-- Logs begin at Tue 2022-09-06 11:07:26 UTC, end at Tue 2022-09-06 12:43:20 UTC. -- Sep 06 11:34:10.579865 dv-cinder-856041-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 11:34:10.588334 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 11:34:10 2022] *** Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: nodename: dv-cinder-856041-1-iscsi Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: machine: x86_64 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: clock source: unix Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: pcre jit disabled Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: detected number of CPU cores: 4 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: current working directory: / Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: detected binary path: /usr/bin/uwsgi-core Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: your processes number limit is 47680 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: your memory page size is 4096 bytes Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: detected max file descriptor number: 2048 Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: lock engine: pthread robust mutexes Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: thunder lock: enabled Sep 06 11:34:10.591970 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 11:34:10.595125 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 11:34:10.632997 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: Python main interpreter initialized at 0x5629f1aea9c0 Sep 06 11:34:10.632997 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: python threads support enabled Sep 06 11:34:10.632997 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: your server socket listen backlog is limited to 100 connections Sep 06 11:34:10.632997 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: your mercy for graceful operations on workers is 90 seconds Sep 06 11:34:10.633424 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 11:34:10.633424 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: *** Operational MODE: preforking *** Sep 06 11:34:10.633793 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: *** uWSGI is running in multiple interpreter mode *** Sep 06 11:34:10.633793 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: spawned uWSGI master process (pid: 87984) Sep 06 11:34:10.634018 dv-cinder-856041-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 11:34:10.637750 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: spawned uWSGI worker 1 (pid: 87996, cores: 1) Sep 06 11:34:10.637750 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: spawned uWSGI worker 2 (pid: 87997, cores: 1) Sep 06 11:34:10.637750 dv-cinder-856041-1-iscsi devstack@keystone.service[87984]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 11:34:14.969508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 11:34:14.969508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: from cryptography.utils import int_from_bytes Sep 06 11:34:14.970338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 11:34:14.970338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: from cryptography.utils import int_from_bytes Sep 06 11:34:14.971097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 11:34:14.971097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: from cryptography.utils import int_from_bytes Sep 06 11:34:14.971551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 11:34:14.971551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: from cryptography.utils import int_from_bytes Sep 06 11:34:15.021696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 11:34:15.021905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 11:34:15.022062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] command line args: [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 11:34:15.022332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 11:34:15.022506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ================================================================================ {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 11:34:15.022759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] admin_token = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] config_dir = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] config_source = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 11:34:15.023526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] debug = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 11:34:15.023725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.023981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] command line args: [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 11:34:15.024049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 11:34:15.024106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ================================================================================ {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 11:34:15.024316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] admin_token = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] config_dir = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.024923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] config_source = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_config_append = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] debug = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_dir = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_file = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.025857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_config_append = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.026940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_dir = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_file = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.027943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.028844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] publish_errors = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.029932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.030816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] standard_threads = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] publish_errors = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] use_journal = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.031985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] use_json = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] use_stderr = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] use_syslog = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] standard_threads = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.032820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.032971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.033048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.033220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.033311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.033403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.033480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.033582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] use_journal = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.033723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.033817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] use_json = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.033954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] use_stderr = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.034038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.034114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] use_syslog = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.034189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.034955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 11:34:15.035193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.035906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.036915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.037946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.038928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.039925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.040883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.041867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.042951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.043950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.044874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.045836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.046905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.047997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.048961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.049965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.050579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.050716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.050821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.050980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.051918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.052974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.053928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.054881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.055977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.056966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.057951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.058870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.external = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.059979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.password = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] auth.token = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.060999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.external = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.061904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.password = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] auth.token = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.062938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.063988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.064895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.065956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.066985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.067912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.068959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.069940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.070853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.071971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.072882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.073956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.074994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.075894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.076950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.077961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.078900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.079917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.080916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.081045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.081792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.081940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082851 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.082977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.083876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.084935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.085986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.086884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.087934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.088972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.089910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.090937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.091891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] role.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.092893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] role.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.093835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.094988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] role.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] role.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.095950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.096719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.097494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.097590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.097776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.097859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.098973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.099982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.100903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.101950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.102925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.103883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.104846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.105865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.connection = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.106858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.107949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.108969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.connection = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.109963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.110739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87997) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 11:34:15.110838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.111079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.111327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.111568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.111817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.112057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.112313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.112810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.112810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.113058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.113278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.113519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 11:34:15.113742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87996) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 11:34:15.141513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.141753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.141985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.143444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.144574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.144810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.151486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.153916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.182225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.182428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.183642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.183880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.241202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.241387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.292586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.292818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.294263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.294502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.451947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.452907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.453009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.453094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.453226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.477739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.477987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.478980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.479227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.479347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.479498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.479498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.479797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.491454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.491658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.504790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.505941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.506070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87997) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.506153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.506305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.506524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.506741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87996) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 11:34:15.522966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.523152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 11:34:15.523502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.523598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.523701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 11:34:15.523785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 11:34:15.523974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 11:34:15.524495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.524817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.525019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.525281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.534862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.535147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.535414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.535743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.535916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.536202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.536476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.536763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.542651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.543814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.545537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.545843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.546127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.546408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.546690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.546773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.546950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.547973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.548113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.548241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.548370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.548647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.548975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.549297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.549624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.550106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.571344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.572802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.573086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.573655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.575146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.575435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.577736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.578015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.578292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.580320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.580604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.580888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.584994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.585276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.587360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.587654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.591060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.591348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.591631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.591912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.592187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.592461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.592730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.592995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.593496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.593830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.594123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.594414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.594699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.594984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.595264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.595540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.609047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.609324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.611901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.612183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.613480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.616313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.616713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.617002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.619582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.619883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.621259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.624362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.624511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.627594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.627743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.627901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.628743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.629188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.629468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.629788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.631679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.631978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.632268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.632551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.632852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.633132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.646172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.646481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.646747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.647004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.650238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.650536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.650812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.651077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.653823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.655371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.656883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.657158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.658006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.659886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.660091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.660255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.660564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.660844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.661240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.661432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.661741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.664416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.664717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.665002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.665287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.665721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.672635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.672933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.673217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.673496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.673914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.674196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.674476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.681331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.681637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.682042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.682332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.682738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.683024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.683310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.691076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.694259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.697187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.698713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.700190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.700470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.700872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.702048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.705169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.707987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.709144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.709483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.710777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.712407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.712558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.712808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.712946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.713106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.713388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.713533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.720230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.721304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.721614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.722482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.722634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.725059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.725638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.726020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.726357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.728936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87997) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.729227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.729505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.729806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.730137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.730337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.730622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.730887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.731149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.731433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.731710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.731968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.732238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.732492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87997) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.735511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.735889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.736237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.743572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87996) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 11:34:15.743839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.744188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.744528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.744873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.745231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.745595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.745973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.746315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.746674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.747017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.747351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.747702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.748040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87996) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 11:34:15.767500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5629f1aea9c0 pid: 87997 (default app) Sep 06 11:34:15.771839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776df8-41e5-4170-b6af-a14aa27e7bd4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:15.772117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776df8-41e5-4170-b6af-a14aa27e7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:15.772389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776df8-41e5-4170-b6af-a14aa27e7bd4 None None] PATH_INFO: `/v3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:15.776631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Tue Sep 6 11:34:15 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:15.789277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5629f1aea9c0 pid: 87996 (default app) Sep 06 11:34:22.992706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:22.993294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:22.993294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:22.994633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:23.005858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:23.006803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:23.006803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:23.142904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: INFO dbcounter [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] Registered counter for database keystone Sep 06 11:34:23.172617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG oslo_db.sqlalchemy.engines [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87996) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 11:34:23.180473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writer thread running {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 11:34:23.379608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG passlib.handlers.bcrypt [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87996) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 11:34:23.380077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG passlib.handlers.bcrypt [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87996) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 11:34:23.443283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:23.445011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:23.620986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 621 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 11:34:24.126199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:24.203288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: INFO dbcounter [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] Registered counter for database keystone Sep 06 11:34:24.230386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG oslo_db.sqlalchemy.engines [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87997) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 11:34:24.247068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writer thread running {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 11:34:24.385013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30fa6-0a07-4c68-bd81-d199409cb354 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:24.389505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:24.390196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:24.390476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:24.390751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:24.401678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:24.456462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 11:34:24.456887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bf9c-5499-45d1-b9bc-65fc93e05e78 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:24.468445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 6 11:34:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:34:24.476052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:24.492501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d8d24-3385-4fd3-a526-5eef4c0a7385 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:24.497483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:24.498692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:24.498978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:24.499254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] PATH_INFO: `/v3/projects/ff9660c7d5b94c33b26aef6fd109810a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:24.500850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] RBAC: Authorizing `identity:get_project(project_id=ff9660c7d5b94c33b26aef6fd109810a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:24.568548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING keystone.common.rbac_enforcer.enforcer [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 11:34:24.568893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c8899-8634-4bb0-876a-b4cd275f8033 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:24.573803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Tue Sep 6 11:34:24 2022] GET /identity/v3/projects/ff9660c7d5b94c33b26aef6fd109810a => generated 344 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:26.726872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dc8de-35db-4c13-88da-89091172b040 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:26.727563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dc8de-35db-4c13-88da-89091172b040 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:26.727563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dc8de-35db-4c13-88da-89091172b040 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:26.731749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:26.738390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:26.739044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:26.739321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:26.762639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:26.764282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:26.773318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 11:34:27.055308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dc8de-35db-4c13-88da-89091172b040 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:27.070003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dc8de-35db-4c13-88da-89091172b040 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:27.075003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:27.075692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:27.076019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:27.076341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:27.086429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:27.088384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bc6db-ecbe-4fa2-a7ac-c86a6f80c305 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:27.099029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 6 11:34:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 11:34:27.105265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:27.116060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2623335-e525-415d-8d35-0281000e2e0d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:27.120923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:27.121683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:27.122021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:27.122350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:27.123948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:27.125982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64f765-ffb0-44eb-bc99-5c746021cee2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:27.127770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Tue Sep 6 11:34:27 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:29.267830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8e041-c1e8-41ed-a3ce-4912a3e71e22 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:29.268387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8e041-c1e8-41ed-a3ce-4912a3e71e22 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:29.268820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8e041-c1e8-41ed-a3ce-4912a3e71e22 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:29.271377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:29.279551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:29.280309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:29.281268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:29.304438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:29.305795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:29.335573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28761e2-0137-410f-a3e1-ea04569ebc4a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:29.336051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28761e2-0137-410f-a3e1-ea04569ebc4a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:29.336523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28761e2-0137-410f-a3e1-ea04569ebc4a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:29.339082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:29.346800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:29.347423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:29.348158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:29.403951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:29.418179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG passlib.handlers.bcrypt [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87997) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 11:34:29.418570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG passlib.handlers.bcrypt [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87997) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 11:34:29.469187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:29.471111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:29.587692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:29.769208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:29.815355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:29.872471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:29.922573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941b7ce-8f45-48d9-a112-89e07c775560 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:29.962343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:29.963142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:29.963519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:29.963911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:29.979903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:29.990115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:29.991926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:29.993441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 6/14] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:30.001353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.033524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fb232-7fe6-43f2-8092-e89e43cecab1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.036979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.081717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.082700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.082987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.083258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.134622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.161178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61544abc-d322-4f5f-8283-45acab4e2c66 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.176329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.178150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.210515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.211305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.211769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.212122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.213062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:30.213442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.215155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.231237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 7/15] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:30.237861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.317599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 9/16] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 11:34:30.324725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.328371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.383273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383eb53-11a8-40c8-960a-c1e84770587a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.437447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.438364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.438769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.439210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.441106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.443880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.446769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 8/17] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:30 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:30.455452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.473973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88971-1cd4-40eb-808e-357ef6ba7075 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.475189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.520777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.521886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.522266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.522684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.546192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.547701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:34:30.570878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:30.572645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:34:30.573723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:34:30.573723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:34:30.573723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:34:30.573723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:34:30.581081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.605315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175e62e-b90a-4966-a37f-868e7e8d62d5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.652071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.652718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.653004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.653272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.654557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.656145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.657994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:30.660620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.667540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:30.725847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab853-66fa-49ca-9103-520ea857cba9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.765245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.765905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.766210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.766483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.767168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:30.767466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.768898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab076d0-455e-4596-8fb8-6c5200c1eb41 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.789773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:30.794718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:30.821276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37056e28-2a19-429c-993a-484f769dc4ec None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:30.863031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:30.864230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:30.864508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:30.864781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:30.865558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.domain.id=default, user.id=f2212262a5dc4cac8d774b88f87a9ec9) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:30.865902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:30.867596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565013-3a9e-4959-9640-e39bf0e4dae9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:30.875869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 6 11:34:30 2022] GET /identity/v3/role_assignments?user.id=f2212262a5dc4cac8d774b88f87a9ec9&scope.domain.id=default&role.id=edae49896b104fddabf7a02644f5d914 => generated 233 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:33.054695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47a58-8c0a-4396-9597-34e3fa24cd89 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.055507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47a58-8c0a-4396-9597-34e3fa24cd89 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.055507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b47a58-8c0a-4396-9597-34e3fa24cd89 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.056631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:33.062833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202a292-93b0-4287-b431-cc15e0a0bb62 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.063599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202a292-93b0-4287-b431-cc15e0a0bb62 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.063599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202a292-93b0-4287-b431-cc15e0a0bb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.094556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8202a292-93b0-4287-b431-cc15e0a0bb62 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:33.095816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8202a292-93b0-4287-b431-cc15e0a0bb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:33.205096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:33.643246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a7c8-41e0-4e08-91d6-3fc7acb2c234 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.643508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a7c8-41e0-4e08-91d6-3fc7acb2c234 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.643889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a7c8-41e0-4e08-91d6-3fc7acb2c234 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.645506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:33.651289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1077cb-86af-4814-b3c5-cbfbd2513ab2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.651568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1077cb-86af-4814-b3c5-cbfbd2513ab2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.651866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1077cb-86af-4814-b3c5-cbfbd2513ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.682936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2c1077cb-86af-4814-b3c5-cbfbd2513ab2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:33.684309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2c1077cb-86af-4814-b3c5-cbfbd2513ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:33.772825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09a7c8-41e0-4e08-91d6-3fc7acb2c234 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:33.807172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:33.937247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad924-31b7-4492-8992-9c3173333d00 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.937929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad924-31b7-4492-8992-9c3173333d00 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.938408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad924-31b7-4492-8992-9c3173333d00 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.940051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:33.947486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:33.948214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:33.948708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:33.975222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:33.998779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:34.000523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:34.035738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09a7c8-41e0-4e08-91d6-3fc7acb2c234 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:34.072160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:34.110601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:34.112225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:34.113099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:34.113907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:34.138778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:34.152993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:34.155003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:34.156739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:34:34.161160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:34.162234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:34.170954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:34.321301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8717-3f3b-4106-be74-9719bc1a620e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:34.374872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:34.375525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:34.375811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:34.376095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:34.376799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:34.377107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:34.378633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:34.381328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:34.395964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 15/29] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:34.402586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:34.532938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:34.596358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbd9d6-46df-42b1-b361-e99ee56a30f1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:34.675750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:34.677718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:34.678007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:34.678261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:34.680334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:34.681884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:34.690309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:34 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:34.697388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:34.790644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129ceca-8355-4881-a728-df0146eb1ea3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:34.858452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:34.871789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:34.872427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:34.872715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:34.872972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:34.873814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:34.875486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:35.037624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf58be-6b1f-45ff-a16f-c39c94aa5794 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:35.038600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 16/31] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 6 11:34:34 2022] POST /identity/v3/roles => generated 225 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:34:35.039653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:35.152020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:35.180424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:35.181526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.182044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.182512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.184843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:35.195795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19943db7-283c-4af6-9e4e-9f2000710590 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:35.199018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 16/32] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:35.202494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2912e4-67ca-4d5d-949d-14b6a54a1d0c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.202883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2912e4-67ca-4d5d-949d-14b6a54a1d0c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.203340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2912e4-67ca-4d5d-949d-14b6a54a1d0c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.205894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 17/33] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:35.207237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2912e4-67ca-4d5d-949d-14b6a54a1d0c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:35.222074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302b76a-5410-4db7-bc18-1bf22897abbd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:35.275724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:35.276649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.277099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.277520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.292029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:35.293992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c355ebe0-2f35-498d-bdee-e44b541afa36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:34:35.312470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:35.313579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:34:35.314642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:34:35.314642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:34:35.314642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:35.316189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:35.319124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.319588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.320027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.358885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:35.360772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:35.419218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2912e4-67ca-4d5d-949d-14b6a54a1d0c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:35.466285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:35.495519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 18/35] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:35.498077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:35.498836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:35.500355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.502493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.502935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] PATH_INFO: `/v3/domains/default/users/f2212262a5dc4cac8d774b88f87a9ec9/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.508135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f2212262a5dc4cac8d774b88f87a9ec9, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:35.513499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249cc0-c0db-436a-b153-686454b9e0b7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:35.590559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:35.696313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 18/36] 10.222.0.18 () {58 vars in 1361 bytes} [Tue Sep 6 11:34:35 2022] PUT /identity/v3/domains/default/users/f2212262a5dc4cac8d774b88f87a9ec9/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:34:35.750311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dda68e-7118-4a35-9d41-58e4fe20be46 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:35.800757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:35.801483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.802068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.802068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.802748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:35.803054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:35.804584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:35.902828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3760d-0eef-4f6b-8a87-45107cd801f1 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.903559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3760d-0eef-4f6b-8a87-45107cd801f1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.904046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3760d-0eef-4f6b-8a87-45107cd801f1 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.909313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 19/37] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:35.917204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311454f-a96c-4db0-a1a3-098a831429ce None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:35.917908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311454f-a96c-4db0-a1a3-098a831429ce None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:35.918395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311454f-a96c-4db0-a1a3-098a831429ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:35.942408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4311454f-a96c-4db0-a1a3-098a831429ce None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:35.945223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4311454f-a96c-4db0-a1a3-098a831429ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:35.962879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 19/38] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:35.968601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:36.046147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 20/39] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:36.122675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:36.123083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5aa6-5651-4ed8-8c9b-90f5828e8237 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:36.123517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5aa6-5651-4ed8-8c9b-90f5828e8237 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:36.123986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5aa6-5651-4ed8-8c9b-90f5828e8237 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:36.126937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 21/40] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:36.132822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b3af6-37ae-4cf4-8c4c-5b568fefff54 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:36.136389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:36.136880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:36.137369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:36.164972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:36.166318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:36.197043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:36.197707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:36.197986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:36.198271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:36.198963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:36.200516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:36.242286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 20/41] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:34:35 2022] POST /identity/v3/projects => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:36.243307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:36.349823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:36.495054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe945f7-5524-4633-9952-c7bbf41e894b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:36.572015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:36.572855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:36.573035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:36.573510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:36.574040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:36.575264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:36.577992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:36.604875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 21/43] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 6 11:34:36 2022] POST /identity/v3/roles => generated 219 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:36.617211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:36.719829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c619-0d37-43d6-b427-909bd334f99e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:36.768068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:36.769086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:36.769741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:36.770242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:36.772167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:36.774333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:36.776407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 23/44] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:36.786486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:36.812779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:36.852609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:36.987383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bd54c-9b22-403d-82fe-a56cec72fb77 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:37.033604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:37.034169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:37.034450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:37.034730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:37.035525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:37.037540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821cc26-8d68-4b4b-b23d-fc0f5ac9871d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:37.039285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:37.044993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17409c6-95af-462d-ad66-a55fb1a8772c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:37.076025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:34:36 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:37.100741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:37.101394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:37.101721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:37.102002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:37.102725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:37.104206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4407117-e2c8-4b4d-a0a0-34fc15be5b49 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:37.129701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 6 11:34:36 2022] POST /identity/v3/roles => generated 223 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:38.254770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730721f-f7da-4a66-a698-da1f1c3831ac None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:38.255549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730721f-f7da-4a66-a698-da1f1c3831ac None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:38.255932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730721f-f7da-4a66-a698-da1f1c3831ac None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:38.260483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:38.267480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:38.268076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:38.268437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:38.295675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:38.296941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:38.388642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:38.680598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730721f-f7da-4a66-a698-da1f1c3831ac None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:38.762205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:38.815882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730721f-f7da-4a66-a698-da1f1c3831ac None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:38.860924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:38.861755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:38.862122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:38.862478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:38.876439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:38.883500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:38.885451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:38.886950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:34:38.893159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:39.012366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:39.025564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0e455-2ce6-48a9-b35d-4a30d0643947 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:39.070567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:39.071297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.071644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.071989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.072776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:39.073278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:39.075003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656a87-007d-4ad7-8a37-c2a0cc20f2b2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:39.090692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 26/50] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:39.099149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:39.222297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f53-9941-45b1-b15c-b2374a282605 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:39.253380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d0102-c177-45c8-bee2-22c7f08e49f4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.253786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d0102-c177-45c8-bee2-22c7f08e49f4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.254167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d0102-c177-45c8-bee2-22c7f08e49f4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.256425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:39.257348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 27/51] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:39.262880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.263397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.263543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.282444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:39.283465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.283930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.284062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:39.284525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.285232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:39.286736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:39.289293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:39.292123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:39 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:39.299403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:39.366199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 28/53] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:39.488738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11bc0a-f6e8-4ce1-8c8d-0ace740ade8d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:39.541314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:39.542092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.542470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.542860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.544762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:39.546654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:39.549071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:39.558800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:39.655215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:39.664999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:39.686447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0b088-c9d6-40f7-9507-cbab2777883b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:39.727678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:39.728304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.728581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.728853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.729603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.domain.id=default, user.id=f2212262a5dc4cac8d774b88f87a9ec9) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:39.729955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:39.731448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:39.740733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 6 11:34:39 2022] GET /identity/v3/role_assignments?user.id=f2212262a5dc4cac8d774b88f87a9ec9&scope.domain.id=default&role.id=edae49896b104fddabf7a02644f5d914 => generated 530 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:39.771845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:39.838889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880de798-9d7b-4bcf-b1f4-69b195a3424b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:39.886647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:39.887439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:39.887810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:39.888171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:39.889606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:39.891281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584388c4-9c81-490d-a026-d1829f093877 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:39.893242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:39.900595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:40.041618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa466790-a0f7-4627-a81f-bc7bde63026f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:40.082076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:40.083396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:40.083886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:40.084341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:40.085092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:40.086696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ee99d-6b61-4433-95d8-5110298f2390 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:40.113261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:40.123837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 6 11:34:39 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:42.272907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772980-cb03-48b4-a37f-7e102b303f59 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:42.273517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772980-cb03-48b4-a37f-7e102b303f59 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:42.273517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772980-cb03-48b4-a37f-7e102b303f59 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:42.275000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:34:42.281189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:42.281760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:42.281879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:42.411578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:42.412840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:42.494072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:42.777896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772980-cb03-48b4-a37f-7e102b303f59 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:42.897534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772980-cb03-48b4-a37f-7e102b303f59 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:42.928178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:42.938245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:42.938907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:42.939189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:42.939470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:42.940673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:42.942822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb3775-f145-4dee-b3e7-9cc7c14ee69a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:42.944645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:42.950819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:43.070662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09193374-930d-47ee-830d-eead8b02e1cb None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:43.102116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:43.111169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:43.111862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:43.112190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:43.112512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:43.113245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:43.115362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c9020-7c98-4be4-ac8e-12e9cd1696af None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:43.146482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:34:42 2022] POST /identity/v3/users => generated 275 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:34:46.716273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7550-e8f6-439a-a905-221182a65236 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:46.716273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7550-e8f6-439a-a905-221182a65236 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:46.716273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7550-e8f6-439a-a905-221182a65236 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:46.716273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:46.722495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994e0c-6377-402e-897d-4c7b3aab4d48 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:46.723217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994e0c-6377-402e-897d-4c7b3aab4d48 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:46.723217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994e0c-6377-402e-897d-4c7b3aab4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:46.750590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-59994e0c-6377-402e-897d-4c7b3aab4d48 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:46.752011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-59994e0c-6377-402e-897d-4c7b3aab4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:46.863503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 33/63] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:47.201698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc528c4c-5f56-43af-9687-2aaf8c5245fd None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.205226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc528c4c-5f56-43af-9687-2aaf8c5245fd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.205226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc528c4c-5f56-43af-9687-2aaf8c5245fd None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.205226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 31/64] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:47.212131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.212520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.212669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.242841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:47.244091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:47.351715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:47.370274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc528c4c-5f56-43af-9687-2aaf8c5245fd None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:47.519470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc528c4c-5f56-43af-9687-2aaf8c5245fd None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:47.573510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:47.574224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.574502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.574780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.591852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:47.596883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:47.598646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:47.600077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:47.602969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 32/66] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:34:47.608403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:47.662598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:47.697139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:47.778314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:47.828182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196fdab-dd0c-4a7e-bb1c-ab9d2c8c10cf None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:47.895657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91495345-900c-4a97-96ca-5720993fb671 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:47.919384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:47.920927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.921622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.922319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.923822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] RBAC: Adding query filter params (name=member) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:47.924704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:47.928565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa35773-afb6-46bb-afe6-c8d61f4481ad None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:47.938199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:47.938859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.939148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.939412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.941589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 35/67] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:34:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:47.944957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb30d-8395-4aa7-8bd4-96d1c0763f1c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.945478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb30d-8395-4aa7-8bd4-96d1c0763f1c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.945972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb30d-8395-4aa7-8bd4-96d1c0763f1c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.948521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 36/68] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:47.952926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026a8b-6185-49bf-82aa-395d4398e59e None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.952926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026a8b-6185-49bf-82aa-395d4398e59e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.952926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026a8b-6185-49bf-82aa-395d4398e59e None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.953222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 37/69] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:47.954632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026a8b-6185-49bf-82aa-395d4398e59e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:47.962028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b961cbf9-86ba-459c-9a83-6610f8a65fb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:47.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:47.969857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 33/70] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:34:47.970959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c594-2947-45e2-9179-da5f4fbcd3e0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:47.971647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c594-2947-45e2-9179-da5f4fbcd3e0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:47.971647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c594-2947-45e2-9179-da5f4fbcd3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:47.995291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1272c594-2947-45e2-9179-da5f4fbcd3e0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:47.996830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1272c594-2947-45e2-9179-da5f4fbcd3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:48.079883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 34/71] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:48.081103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.081393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.081696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.099479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026a8b-6185-49bf-82aa-395d4398e59e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:48.104112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:48.105685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:48.111612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.155902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.179957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:48.180951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.181417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.181911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.209271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 35/72] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:48.209517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:48.212198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126dd71-fc28-466d-a08f-78ff71ef388a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:48.215540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 38/73] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:47 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:48.216376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:48.218485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8086c02-2981-484b-adf8-e1440106d46c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.219322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8086c02-2981-484b-adf8-e1440106d46c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.220080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8086c02-2981-484b-adf8-e1440106d46c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.222950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 39/74] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:48.234157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8086c02-2981-484b-adf8-e1440106d46c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:48.446621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925b20e-b441-4fb7-a15f-a4aeb548fd16 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:48.467081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8086c02-2981-484b-adf8-e1440106d46c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:48.511750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:48.512827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.513315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.514002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.515125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:48.515670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:48.518339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb160e4-30ae-482e-a27a-28d8c94c9da1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:48.539359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 36/75] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:48.539359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.539359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.539359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.541524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:48.542616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.543117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.543735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.569328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:48.571159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:48.588850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.623681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.644706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:48.646328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:48.688581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 37/76] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:48.694140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:48.788251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 40/77] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:34:48.788251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:48.862431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024040d-d7cf-4741-a280-6d6e7f4e6a29 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:48.866041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.911890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:48.912749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.913106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.913462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.915377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d486-9b42-445c-b18a-dcc8b098df6c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:48.928104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:48.933323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:48.934429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:48.935322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:48.937378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:48.941059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:48.976086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:48.976803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:48.977087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:48.977384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:48.977630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:48.979142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:48.980764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:48.987475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 41/79] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:48.987475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.106073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b9f1-a380-4a09-add0-9fb310c95671 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.116292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee35c0-d4a2-4e74-85ce-2def75e6e5d9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.156249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.156897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.157180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.157451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.164045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:49.171140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:49.171388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.172050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.172329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.172603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.173917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.175410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:49.177192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:49.181944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:34:49.191998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:49.193946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 42/80] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:49.194086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 39/81] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:34:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:49.194311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.194475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.314415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:49.334314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e952e3-5482-4ca1-a77c-b04ca7482530 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.342334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2261cfe-fc6d-4e08-9c27-9edc0e7ad26a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.387970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.388846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.389299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.389744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.398937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.400060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.400580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.401167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.402595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] RBAC: Adding query filter params (name=member) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:49.403098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.405443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.418770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 43/82] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:49.420063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.491117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.493981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.501623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:49.549346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284dfd-92cd-41e6-bee6-10a45c760e15 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.590654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.591275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.591565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.591841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.592527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:49.592834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.594286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.604423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 44/83] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:49.605320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.651103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:49.674314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 40/84] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:34:49.675103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.752063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bab20-858a-432b-8609-9ac05125dcc4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.787712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:49.804906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.805876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.806292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.806759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.807659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:49.808004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.809548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.817217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 45/85] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=0da2f51a570245b1acf832a45de984cc => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:49.818528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.827085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee41a1-bef3-48c2-ba44-223b52eac203 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:49.905522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:49.906522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:49.907711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:49.908176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:49.911330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:49.915485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:49.919107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 41/86] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:49.925847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:49.992555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d70899-d749-4aa2-bbc4-2c17fa8eb1bb None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.008784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:50.068387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.069349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.069869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.070352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.072462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.075202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.078278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 46/87] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:50.079758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bec-d00b-48aa-970b-53b753135ce8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.081323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:50.125460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.126164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.126457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.126738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.128011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.131059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.132537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 42/88] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:49 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:50.133363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:50.171175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:50.237631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6c63-9581-42e4-81b7-d171719de5f9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.273867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd456-358e-42af-aa49-249ee5f6dbae None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.292388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.293396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.293874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.294239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.295764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.297538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.305757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 47/89] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:50 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:50.305757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:50.311169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:50.328383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.329494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.329983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.330574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.331841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.334517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.385703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 43/90] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:34:50 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:50.385703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:50.493701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688471fb-6127-4c44-862d-2cde10394c7d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.543555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:50.574073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.575154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.575653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.576164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.601629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214640-80f2-400e-8c85-2f0deb66e2bc None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:50.651926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:50.653021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:50.653604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:50.654093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:50.735775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.737126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.762490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:50.816370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:50.819725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:50.987376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 48/91] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:34:50.989049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:51.027168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 44/92] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:34:51.028572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:51.039587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:51.191086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:51.237282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d144f6-c53a-4450-b04b-54e5851306a3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:51.305520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:51.306697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:51.307280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:51.312321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] PATH_INFO: `/v3/projects/20c5fb40a3354579b3a3d653e031bdc5` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:51.312321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] RBAC: Authorizing `identity:get_project(project_id=20c5fb40a3354579b3a3d653e031bdc5)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:51.312321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:51.315533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 45/93] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:51 2022] GET /identity/v3/projects/20c5fb40a3354579b3a3d653e031bdc5 => generated 312 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:51.316880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:51.325051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c011239-21ea-4a68-9e6a-4bdac4b241a7 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:51.457708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:51.458753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:51.459343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:51.461195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:51.462617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=f2212262a5dc4cac8d774b88f87a9ec9) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:51.463154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:51.469698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:51.477411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 49/94] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:34:50 2022] GET /identity/v3/role_assignments?user.id=f2212262a5dc4cac8d774b88f87a9ec9&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:51.478908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:51.697097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab098bf-6b69-44ae-9ba4-7998ae6e1d98 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:51.750718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:51.804061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca031dd-3986-4a87-98f4-869503cbec12 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:51.829477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:51.830559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:51.831040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:51.831487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:51.839275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:51.842249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:51.845571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 46/95] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:51 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:51.862678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:51.918492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:51.944930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:51.946001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:51.946666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:51.947196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:51.948354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=20c5fb40a3354579b3a3d653e031bdc5, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:51.948841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:51.951704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11799c-55e4-447e-8aff-7920fac15495 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:51.969782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 50/96] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:34:51 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=20c5fb40a3354579b3a3d653e031bdc5&role.id=0da2f51a570245b1acf832a45de984cc => generated 259 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:52.165543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d4be1-7e92-4051-a878-059413ad53a7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:52.244007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:52.244952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:52.245434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:52.249693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:52.249693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:52.249693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:52.251027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd099e8-c39d-4b94-8d45-a0e91cee3de4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:52.268194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 47/97] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:34:51 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 259 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:53.608578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fa2c-2643-48bc-bb40-58fc193c9c85 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:53.609323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fa2c-2643-48bc-bb40-58fc193c9c85 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:53.609475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fa2c-2643-48bc-bb40-58fc193c9c85 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:53.612301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 51/98] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:53.619875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37068e4-e437-43ad-af67-eb59bbaad03d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:53.620649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37068e4-e437-43ad-af67-eb59bbaad03d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:53.620777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37068e4-e437-43ad-af67-eb59bbaad03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:53.662544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e37068e4-e437-43ad-af67-eb59bbaad03d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:53.664645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e37068e4-e437-43ad-af67-eb59bbaad03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:53.810831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 48/99] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:54.425894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fa2c-2643-48bc-bb40-58fc193c9c85 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:54.566713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa701f6-c29f-41c9-8213-b7e0c5961f08 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:54.567954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa701f6-c29f-41c9-8213-b7e0c5961f08 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:54.568123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa701f6-c29f-41c9-8213-b7e0c5961f08 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:54.571006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 49/100] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:54.577708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:54.578143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:54.578617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:54.634868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:54.636978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:54.641041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:54.798674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fa2c-2643-48bc-bb40-58fc193c9c85 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:54.893132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 50/101] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 06 11:34:54.906275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:54.907613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:54.908226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:54.908830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:54.930957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-f0f67814-6432-49f1-9cb2-19045ba0ceaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:54.942199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:54.943139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:54.943908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:34:54.944943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 52/102] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:54.954436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:55.044289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a822c07-1046-461c-b31e-811320b85a95 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:55.049242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a822c07-1046-461c-b31e-811320b85a95 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:55.049242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a822c07-1046-461c-b31e-811320b85a95 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:55.049242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 53/103] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:55.057712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:55.061674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:55.061674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:55.118900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:55.120547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:55.145056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:55.307167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 54/104] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:34:55.377033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf625-1c71-4d31-b49d-850dde1696af None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:55.485338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:55.486390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:55.486880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:55.487354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:55.488571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:55.489706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:55.492363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:55.507109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:55.518172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 51/105] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:34:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 565 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:55.524883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:55.706303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5ebc3-0e8e-47a1-b62f-81d5835d64da None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:55.725880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:55.757487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:55.758421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:55.758850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:55.759267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:55.761262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:55.767492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:55.769624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 55/106] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:55 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:55.775693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:55.806228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:55.927805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f96196-e268-40e8-b4cd-3004c1d5a968 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:55.973007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:55.974897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:55.977677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:55.977677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:55.979718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:55.981224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:55.983290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 56/107] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:55.983788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:55.992267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a43467-cb56-428f-a5ad-ef9bcaa8e8c7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.087195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:56.087756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.088033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.088315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.129035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:56.151284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:56.153037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:34:56.154683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:56.156560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 52/108] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:56.156560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:56.174205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:56.197162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf15e-73b4-400f-8f07-7b7846405314 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.240303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:56.240979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.241265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.241549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.242953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:56.244638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:56.246493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 57/109] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:55 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:56.247477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:56.382962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2622c-46ea-47d2-b065-5eaf4fa8cc26 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.400478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:56.460130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:56.460771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.461057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.461334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.462524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:56.463059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:56.465678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:56.480727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 58/110] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:34:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:56.482076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:56.496938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05fe0a-b0a3-4ef0-8d0d-67bb4a44dda1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.507260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:56.566542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:56.568822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.569685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.570478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.571952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:56.575020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7328-fec1-4de7-8eab-f65598f7f2fe None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:56.631471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58e92-8229-471a-b621-3c47cec903f5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.632214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 53/111] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:34:56 2022] POST /identity/v3/projects => generated 302 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:34:56.635630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b1e1-e6d3-47e2-8ff9-c78d346a862e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.636748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b1e1-e6d3-47e2-8ff9-c78d346a862e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.638012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b1e1-e6d3-47e2-8ff9-c78d346a862e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.641908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 54/112] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:56.644050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53b1e1-e6d3-47e2-8ff9-c78d346a862e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:56.714370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:56.716200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.717145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.718011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/2c22fa2cde544208ab6c0e4a6f861028/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.722722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user_id=2c22fa2cde544208ab6c0e4a6f861028, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:56.727011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa69f6-bb5b-4e43-82e6-efc5476a75e4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:56.761688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 59/113] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:34:56 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/2c22fa2cde544208ab6c0e4a6f861028/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:34:56.761688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55f37-3372-4fbe-99ee-c70deb02f061 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.761688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55f37-3372-4fbe-99ee-c70deb02f061 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.761688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55f37-3372-4fbe-99ee-c70deb02f061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.796183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-14b55f37-3372-4fbe-99ee-c70deb02f061 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:56.798944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-14b55f37-3372-4fbe-99ee-c70deb02f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:56.948461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53b1e1-e6d3-47e2-8ff9-c78d346a862e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:56.951617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:56.961706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 60/114] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:56.963680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff26ab-ea83-4275-aa42-295b4d6c2c0c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.964151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff26ab-ea83-4275-aa42-295b4d6c2c0c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.964946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff26ab-ea83-4275-aa42-295b4d6c2c0c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.967014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 61/115] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:56.976610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:56.977140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:56.977611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:56.985490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:57.000607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:57.002314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:57.056290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:57.057440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:57.059341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:57.060055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] PATH_INFO: `/v3/users/c185b65860914a77acb771ce1045843c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:57.090578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] RBAC: Authorizing `identity:get_user(user_id=c185b65860914a77acb771ce1045843c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:57.094009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da925f-c633-436b-914e-8451bf5f9edd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:57.097008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 55/116] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:56 2022] GET /identity/v3/users/c185b65860914a77acb771ce1045843c => generated 289 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:57.098812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ac0a-1fee-4432-9a14-2907ec468773 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:57.099391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ac0a-1fee-4432-9a14-2907ec468773 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:57.099884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ac0a-1fee-4432-9a14-2907ec468773 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:57.101519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 56/117] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:34:57.110302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:57.111244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:57.111820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:57.137184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 62/118] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:57.138715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:57.139042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:34:57.140325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:34:57.239706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 57/119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:34:57.369554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd207d9-d385-4c71-a65f-dc124f446eb2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:57.428170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:57.442037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:57.443855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:57.444316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:57.444760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:57.459549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:57.632001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:57.635387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:57.696884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:57.703460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe879db-76d7-40d2-aa0a-9c741dd6233e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:57.798199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:57.799335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:57.800004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:57.800637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:57.845664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:57.857319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 63/120] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:34:57.858246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:57.867574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:57.869104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:57.871348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 58/121] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:57.881553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:57.898707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.002766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545089-b803-49c2-b19e-3fde9e151391 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.071187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.072502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.073018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.074016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.094232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c088-2a09-489c-a808-04f8cc236208 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.099123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:58.111769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:58.113765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:34:58.116515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 64/122] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:58.118502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.151339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.152241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.152688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.153030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.153843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:58.154214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.155775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.168143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 59/123] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:34:58.169239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.205753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.307354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92dce4-6070-4b93-a295-dd76ce0ce719 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.316902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.345764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817953ad-d56e-426f-8ecf-1933a819cf49 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.352852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.353845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.354303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.354723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.378357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:34:58.393440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:34:58.395483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 60/124] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:34:58.397938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.430612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.432047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.432616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.433301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.435834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.439261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.442971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 65/125] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:58.446503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.499744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.524236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5db6-0160-44ed-aadc-a44bebeb1c56 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.563735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.564455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.564739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.565019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.565946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:58.566261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.568569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.595739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 61/126] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:58.597433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.643131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394fb2-3f84-4188-9218-5035374bd6b0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.694573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.695688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.695948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.696307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.697625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.699188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.701822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 66/127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:58.702759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.742918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.803328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d63e-a7f0-4c24-88fd-eba827e33178 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.869582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.870670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.871162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.871641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.872853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:58.873439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.876131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.885579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59dd8d-c8c3-483b-8142-40097bc4d53a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:58.898457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 62/128] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:58.898457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:58.917941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:58.962776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:58.964548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:58.965334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:58.966072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:58.968180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=c185b65860914a77acb771ce1045843c) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:34:58.969016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:58.973298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:58.986561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 67/129] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/role_assignments?user.id=c185b65860914a77acb771ce1045843c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:58.988351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:59.183659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:59.209118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817c5b-5c2f-400d-947a-b6530df0eabb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:59.214657 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcbdc4-4af7-45a2-8bb6-1e4a3d785e0b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:59.278184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:59.278928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:59.279203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:59.279455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:59.282094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:59.283621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:59.285438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 68/130] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:59.287227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:59.335213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:59.335880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:59.336161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:59.336434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:59.342738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:59.344283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:59.346078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 63/131] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:58 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:59.350973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:59.463161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:59.523829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e502-e05c-492d-879b-51553cb14b71 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:59.531285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4d10-7524-41e8-9886-41a2a6b004d0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:34:59.606843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:59.609750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:59.610984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:59.613066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:59.618568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:59.624155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:59.633331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 69/132] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:34:59 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:34:59.636862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:59.653890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:34:59.657295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:34:59.659715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:34:59.664188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/f2212262a5dc4cac8d774b88f87a9ec9/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:34:59.674171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user_id=f2212262a5dc4cac8d774b88f87a9ec9, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:34:59.678284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:34:59.726779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:34:59.727062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 64/133] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:34:59 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/f2212262a5dc4cac8d774b88f87a9ec9/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:34:59.729211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:34:59.971270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311cf3a-68a9-45a5-a51b-dfb6c3bdb242 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:00.023978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de08edf-a21f-49aa-8f0b-a3263aea418c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:00.055597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:00.056717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.057203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.057716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] PATH_INFO: `/v3/projects/20c5fb40a3354579b3a3d653e031bdc5` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.059955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] RBAC: Authorizing `identity:get_project(project_id=20c5fb40a3354579b3a3d653e031bdc5)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:00.063018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:00.069532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 65/134] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:59 2022] GET /identity/v3/projects/20c5fb40a3354579b3a3d653e031bdc5 => generated 312 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:00.078818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:00.086011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:00.127348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:00.129050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.129947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.138179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.143433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:00.149212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:00.162644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 70/135] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:34:59 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:00.172748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:00.306776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:00.338486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ba9b-1c74-4d11-bfa1-756716209555 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:00.418020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb782b4-4966-4ee7-804b-bacab10a3412 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:00.432712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:00.434189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.434909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.435712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/2c22fa2cde544208ab6c0e4a6f861028/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.439835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user_id=2c22fa2cde544208ab6c0e4a6f861028, role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:00.444429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159da38-8759-49ce-8832-18b7b856f8fb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:00.474999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 71/136] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:00 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/2c22fa2cde544208ab6c0e4a6f861028/roles/acebbc3107e1448599b7b2bd0096b755 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:00.536723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:00.537782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.538264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.538746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] PATH_INFO: `/v3/projects/20c5fb40a3354579b3a3d653e031bdc5/users/2c22fa2cde544208ab6c0e4a6f861028/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.546285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5fb40a3354579b3a3d653e031bdc5, user_id=2c22fa2cde544208ab6c0e4a6f861028, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:00.549062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e936-c464-4d59-84eb-63cdb694db4b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:00.582262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 66/137] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:00 2022] PUT /identity/v3/projects/20c5fb40a3354579b3a3d653e031bdc5/users/2c22fa2cde544208ab6c0e4a6f861028/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:35:00.741478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340835-202a-4c56-b65e-3e865b10cbeb None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.742580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340835-202a-4c56-b65e-3e865b10cbeb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.743266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340835-202a-4c56-b65e-3e865b10cbeb None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.746531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 72/138] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:00.755885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.756331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.756917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.798391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:00.800364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:00.910021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689797-4539-4792-83a6-4eb5991cc314 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.910680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689797-4539-4792-83a6-4eb5991cc314 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.911199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689797-4539-4792-83a6-4eb5991cc314 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.914531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 73/139] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:00.920708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:00.921344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:00.922598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:00.975313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:00.977546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:00.990885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:01.032571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 67/140] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:01.149270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 74/141] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:01.663656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:01.725213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:01.829223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0fdd3-3f38-4bbf-addf-94040591f1fe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:01.883688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:01.884361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:01.884643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:01.884919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:01.888241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:01.889793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:01.891697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 68/142] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:01.910935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:02.009156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:02.048308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:02.075146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1bb47-0045-4142-a2fb-89290ae96f4c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:02.123253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:02.124358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:02.126050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:02.127166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:02.129255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd6f9a-a662-4c49-b937-25b23b067fa7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:02.130870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:02.134965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:02.186681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 69/143] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:01 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:35:02.204465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:02.205454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:02.205929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:02.206460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:02.220963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:02.227390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:02.228534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:02.229438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:02.231610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 75/144] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:02.241912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:02.454060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1567a-880b-49a8-88e9-23b75dc9154e None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:02.494061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:02.551130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:02.552927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:02.553767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:02.554278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:02.555559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:02.556095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:02.558721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:02.581036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 70/145] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:02.589366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:02.739501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8c03f-1678-42db-a404-e34ac3d53e38 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:02.789572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:02.790653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:02.791144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:02.791623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:02.793165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:02.794933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf795c36-3ca2-43db-817a-33113ac2b933 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:02.797263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 76/146] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:02 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:02.808730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:02.980927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70dc3e-11a2-439c-946d-fd225d9685f1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:03.029560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:03.030239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:03.030517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:03.030779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:03.085071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d4da-b4d4-4b8d-8eb6-3642f7352924 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:03.086264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d4da-b4d4-4b8d-8eb6-3642f7352924 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:03.087060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d4da-b4d4-4b8d-8eb6-3642f7352924 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:03.094799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 77/147] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:03.102116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:03.103156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:03.103921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:03.104611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:03.159229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:03.162625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:03.162839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:03.164015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:03.169941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:03.374154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 78/148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:03.464058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 71/149] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:03.471901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:03.726646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57170-7adc-45a8-b2e6-386350af5ec9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:03.813229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:03.815547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:03.816296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:03.817014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:03.822988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:03.826080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:03.829576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 79/150] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:03 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:03.839913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:04.055402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:04.106950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37433b32-61bd-41a0-be5b-d8a3764b8e8b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:04.184390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:04.185391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:04.185884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:04.186355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:04.187718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:04.188254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:04.191021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d0a59-f7a8-4ac6-b3ac-e78dd98003f0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:04.203176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 72/151] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:03 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=0da2f51a570245b1acf832a45de984cc => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:04.364780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:04.388972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:04.477003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b05e-9772-450c-bb7f-5057827d4850 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:04.477971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b05e-9772-450c-bb7f-5057827d4850 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:04.478771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b05e-9772-450c-bb7f-5057827d4850 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:04.488517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 73/152] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:04.501928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:04.502831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:04.503588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:04.554783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:04.557222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:04.590493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada4819-7367-4cfa-930b-5787a080b36b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:04.669236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:04.670770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:04.671484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:04.672152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:04.703326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:04.715489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:04.717116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:04.718632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:04.722916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 80/153] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:04.732599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:04.746577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 74/154] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:04.871714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:04.942098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd738e-0c5b-4f6d-836b-16a889f53fab None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:05.021391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:05.022493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.022974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.023444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.024668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:05.025194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:05.027859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdd1c-299c-482a-9de6-2ad5d7db29d1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:05.044735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 81/155] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:05.054090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:05.224299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:05.296285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191fba-6bcb-4a04-9aca-142a0904949a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.296573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e543c6e-a30a-434d-ae9a-e1dfba4182c4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:05.297345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191fba-6bcb-4a04-9aca-142a0904949a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.298092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191fba-6bcb-4a04-9aca-142a0904949a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.301162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 82/156] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:05.326573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1be3-7255-43ef-8e25-35d29115e91d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.327707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1be3-7255-43ef-8e25-35d29115e91d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.328432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1be3-7255-43ef-8e25-35d29115e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.391442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-54df1be3-7255-43ef-8e25-35d29115e91d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:05.394064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-54df1be3-7255-43ef-8e25-35d29115e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:05.424885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:05.426615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.428775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.429609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] PATH_INFO: `/v3/users/c185b65860914a77acb771ce1045843c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.432208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] RBAC: Authorizing `identity:get_user(user_id=c185b65860914a77acb771ce1045843c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:05.435466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:05.443383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 75/157] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:05 2022] GET /identity/v3/users/c185b65860914a77acb771ce1045843c => generated 289 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:05.452712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:05.661045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 83/158] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:05.728419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255fdaf-4df8-43ad-ae49-51db76ff57ee None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.728765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255fdaf-4df8-43ad-ae49-51db76ff57ee None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.729080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255fdaf-4df8-43ad-ae49-51db76ff57ee None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.730455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 84/159] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:05.752214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255fdaf-4df8-43ad-ae49-51db76ff57ee None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:05.784933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8801c-2c74-4951-b1ae-bd12fb523ffa None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:05.903772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:05.905358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.906175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:05.906901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.906901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.915219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:05.918160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053cf293-0be3-4892-9a76-2543fdf15839 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:05.921848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 76/160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:05 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:05.923831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:05.924297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:05.924765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:05.996991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:06.000890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:06.008399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:06.027803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255fdaf-4df8-43ad-ae49-51db76ff57ee None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:06.178776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:06.180258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:06.180740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:06.181206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:06.219479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:06.232216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:06.234073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:06.236013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:06.241203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 85/161] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:06.246857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:06.310900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 77/162] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:06.313494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:06.491991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adc4a6-d3f1-4988-9924-41b82275a19b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:06.521887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:06.538299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:06.538990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:06.539273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:06.539664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/c185b65860914a77acb771ce1045843c/roles/5af87c99a4b241e7b310b566fca557c5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:06.541942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user_id=c185b65860914a77acb771ce1045843c, role_id=5af87c99a4b241e7b310b566fca557c5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:06.543467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:06.561262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 86/163] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:06 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/users/c185b65860914a77acb771ce1045843c/roles/5af87c99a4b241e7b310b566fca557c5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:06.562222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:06.706265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dd3a6-4de6-423f-8531-36ddc2f0d1ed None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:06.730929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc80458-34a2-40bf-8e03-8c4185e34ba5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:06.777075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:06.777636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:06.777960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:06.778227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:06.778942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:06.779315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:06.780833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:06.792422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 87/164] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:06.800860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:06.805534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:06.806250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:06.806523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:06.806803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:06.824669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:06.838780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:06.843917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:06.844879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:06.845697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 78/165] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 533 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:06.853624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:06.879790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:06.958859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a93b00-9bc8-4033-ae6f-5d8c16eb2661 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.001424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:07.002134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:07.002416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:07.002693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] PATH_INFO: `/v3/users/f2212262a5dc4cac8d774b88f87a9ec9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:07.004041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] RBAC: Authorizing `identity:get_user(user_id=f2212262a5dc4cac8d774b88f87a9ec9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:07.005702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:07.007701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 88/166] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:06 2022] GET /identity/v3/users/f2212262a5dc4cac8d774b88f87a9ec9 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:07.014251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:07.137627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1726a8-834d-44bf-9680-73c2141ca98b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.150923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:07.183701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:07.184580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:07.184863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:07.185136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:07.237160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b1891-6567-4799-994c-13a3a2ee6f2f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.272154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:07.274142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:07.339255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:07.341049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:07.341610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:07.342156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:07.343354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:07.343906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:07.345591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:07.366765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 79/167] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:07.368088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:07.406800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 89/168] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 11:35:07.412381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:07.428920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:07.546231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:07.574945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12fb77-6c41-49a2-915d-cf451eb48360 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.657364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:07.658425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:07.659616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:07.660385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:07.662842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:07.665770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:07.670061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 90/169] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:07 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:07.671738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:07.706137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c853d-f8a3-4c85-911a-bb9daef0a4fc None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.870897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:07.871972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:07.872454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:07.872926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:07.893513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:07.934776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:07.938934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeed9-4ec7-4e3b-9730-2db3bb66dfe1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:07.945529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:07.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:07.948694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:07.956251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 80/170] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 588 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:07.957789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:08.024980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:08.026386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.026888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.027965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.030618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:08.033727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:08.038582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 91/171] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:07 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:08.043289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:08.281712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53c3fa-4b83-48df-aef4-a42dce052f29 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:08.297631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584203-6c7f-4c16-abe5-f64e200fa3c5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:08.368818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:08.369932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.370400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.370892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.372194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:08.372746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:08.382574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:08.386254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8e244-4164-4ff2-a6b2-601ef3d35453 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:08.420272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 92/172] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:08.423154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe433-6eaf-4bbe-aed7-3eb584687177 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.423659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe433-6eaf-4bbe-aed7-3eb584687177 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.424155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe433-6eaf-4bbe-aed7-3eb584687177 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.428255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 93/173] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:08.429855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfe433-6eaf-4bbe-aed7-3eb584687177 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:08.443002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:08.444162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.444646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.445120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.453386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:08.652892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:08.655605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dcfa0-f739-4873-b1b0-5f6609c13020 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:08.752733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfe433-6eaf-4bbe-aed7-3eb584687177 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:08.827246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:08.828322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.828834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.829330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.830799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=f2212262a5dc4cac8d774b88f87a9ec9) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:08.831362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:08.834770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:08.853046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 94/174] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:08 2022] GET /identity/v3/role_assignments?user.id=f2212262a5dc4cac8d774b88f87a9ec9&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:08.853350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 81/175] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 891 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:08.854075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:08.854210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ccb2-26b4-41d2-b2b4-d917c98aba3c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:08.854210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ccb2-26b4-41d2-b2b4-d917c98aba3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:08.854210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ccb2-26b4-41d2-b2b4-d917c98aba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:08.875897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9831ccb2-26b4-41d2-b2b4-d917c98aba3c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:08.877257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9831ccb2-26b4-41d2-b2b4-d917c98aba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:08.897521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:09.006655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:09.014005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 82/176] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:09.020379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273dd0e-1c8a-454f-87d2-73977dbb1bee None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.022054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273dd0e-1c8a-454f-87d2-73977dbb1bee None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.022886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273dd0e-1c8a-454f-87d2-73977dbb1bee None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.026069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 83/177] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:09.037783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c27f9-332b-44d5-a36e-a7cdbe1360f7 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.038798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c27f9-332b-44d5-a36e-a7cdbe1360f7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.039645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c27f9-332b-44d5-a36e-a7cdbe1360f7 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.043736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 84/178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:09.045758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c27f9-332b-44d5-a36e-a7cdbe1360f7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:09.103506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e928d31-c6e8-4afd-bc09-911554bc179f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:09.183840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:09.184617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.184920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.185194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] PATH_INFO: `/v3/users/2c22fa2cde544208ab6c0e4a6f861028` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.187795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] RBAC: Authorizing `identity:get_user(user_id=2c22fa2cde544208ab6c0e4a6f861028)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:09.189544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00ab0-471a-456f-ad2a-0ad7bc4b6a91 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:09.191812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 95/179] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:08 2022] GET /identity/v3/users/2c22fa2cde544208ab6c0e4a6f861028 => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:09.194952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.195322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.195586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.200744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c27f9-332b-44d5-a36e-a7cdbe1360f7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:09.230200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:09.231434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:09.265603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:09.266229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.266508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.266776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] PATH_INFO: `/v3/projects/20c5fb40a3354579b3a3d653e031bdc5` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.268024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] RBAC: Authorizing `identity:get_project(project_id=20c5fb40a3354579b3a3d653e031bdc5)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:09.269536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca559c-73e3-410f-8e2f-3f7016196342 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:09.271979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 85/180] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:09 2022] GET /identity/v3/projects/20c5fb40a3354579b3a3d653e031bdc5 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:09.273926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.274178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.274519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.301264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:09.303444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:09.324074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:09.413408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 86/181] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:09.415700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:09.433033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 96/182] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:09.435118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:09.654550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e8265-d7b4-4cc9-a11d-0b78e6772d91 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:09.676758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:09.716196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f2035-30d1-4924-a5c5-0c161163383e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:09.751488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:09.752583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.752906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.753311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.763743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:09.764747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:09.765224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:09.765765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:09.766742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=20c5fb40a3354579b3a3d653e031bdc5, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:09.768317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:09.770235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:09.785952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 97/183] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:09 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=20c5fb40a3354579b3a3d653e031bdc5&role.id=0da2f51a570245b1acf832a45de984cc => generated 608 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:09.788686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:09.802800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:09.894810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:09.897342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4eb7da-ad6d-4a8c-ac95-34d95d39e54f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:10.022535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 87/184] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:10.023746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d158c09-9c69-4a50-85e5-d2439e24d806 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.024158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d158c09-9c69-4a50-85e5-d2439e24d806 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.024402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d158c09-9c69-4a50-85e5-d2439e24d806 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.025167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cfec-1047-4c8b-b489-17edc6c2c0ea None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.026008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 88/185] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:10.030881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d158c09-9c69-4a50-85e5-d2439e24d806 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.090925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.092034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.092519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.093004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.096669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.110069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.119502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:10.130488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:10.132346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:10.133614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 98/186] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:10.134941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.214676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d158c09-9c69-4a50-85e5-d2439e24d806 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.255728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.256556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.257208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.257684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.274197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:10.282467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:10.285317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 89/187] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:10.287423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.306053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f242d5-db72-4a78-81e8-82826d607991 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.361987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.362970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.363417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.363849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.365508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:10.367210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ed25f-58e2-482b-af01-eb77315d6fe8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:10.369088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 99/188] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:10.370756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.371218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.371616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.394275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:10.395706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:10.437176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe056-4992-4f01-8be6-6d883297c8d9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.458008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.460263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.481327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 100/189] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:10.482267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.507334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.507991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.508266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.508540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.509807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:10.511310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:10.513072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 90/190] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:10.522293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.648312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655aea84-59bd-4bdc-89d0-ed1add0aa286 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.691965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9da13-f0f9-4497-b1dc-8581f44d8938 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.716528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.717582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.718107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.718637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.719867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:10.720379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:10.723030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:10.741732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 101/191] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:10.741977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.749725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.749971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.750148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.750148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:10.750148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:10.750148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:10.750148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:10.769256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 91/192] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:10.773126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:10.840615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.848669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:10.936073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570d9f0-4e40-4935-982a-941280d8ae9b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.963285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527386c-a10a-4516-bf25-116804e03934 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:10.998164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:10.999226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:10.999581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:10.999966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.000741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.002594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.004625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.005962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.006239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.006511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.007510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=2c22fa2cde544208ab6c0e4a6f861028) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:11.007813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.009338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.016626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 92/193] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:10 2022] GET /identity/v3/role_assignments?user.id=2c22fa2cde544208ab6c0e4a6f861028&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:11.017603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.046496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 102/194] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:10 2022] POST /identity/v3/users => generated 297 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:11.059622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.186682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210e640-3a8c-4d02-b733-9ffa61373b38 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:11.189438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:11.245683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.246861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.247359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.247838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.276019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.277948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.279915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 93/195] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:11.281019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.281475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d126e73-ea5c-4181-b44d-7c81e0931b9e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:11.306716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:11.356109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.356869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.357196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.357528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.359870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.360635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.363530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 103/196] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:11.363530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.478785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d1df5-5bb6-47cc-b559-62a50e12c5b4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:11.547504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.548543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.548988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.549428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.572060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:11.579454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f511-0861-4f1f-9c39-d1793a608526 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:11.584208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:11.585362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:11.586401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 94/197] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:11.587930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.628207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:11.629226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.630425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.630989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.631530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.695421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:11.726216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.727733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.758895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d8469-08fe-43d3-b319-ef6a5f118d7b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:11.805116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:11.805794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:11.806071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:11.806341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:11.874795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 104/198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:11.876641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:11.900720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:11.902529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:11.929323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:11.998077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:12.040706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0d566-9ecb-4738-a8d7-1a5d0a3f3f3d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.055578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 95/199] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:12.055578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.096923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.097816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.098123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.098513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:12.099354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:12.099835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:12.101380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:12.110866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 105/200] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:12.111723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.204036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b77e-8403-4dea-b6ff-a937445e7aff None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.234678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f20b-767a-4182-a384-440e13f6f964 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.254400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:12.275601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.277018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.277764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.278462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:12.281968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:12.286025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:12.291996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 96/201] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:12.298016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.327576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.332318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.334112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.335876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:12.338964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:12.345286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:12.351785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 106/202] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:12.353620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.382519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:12.534938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c22-eb94-4140-b186-191087d484a3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.568813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe84658-7892-4026-9115-b8e15ea360f3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.609715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.610627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.610895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.611291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] PATH_INFO: `/v3/users/c185b65860914a77acb771ce1045843c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:12.612668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] RBAC: Authorizing `identity:get_user(user_id=c185b65860914a77acb771ce1045843c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:12.614555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:12.616619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 97/203] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/users/c185b65860914a77acb771ce1045843c => generated 289 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:12.622950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.639996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.641552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.642298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.643205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:12.645142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=43338d760135414e9fb6716858b0858f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:12.647724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:12.651009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:12.669963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 107/204] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/role_assignments?user.id=43338d760135414e9fb6716858b0858f&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:12.671967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:12.754474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:12.839125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:12.879254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7383f-d67e-469e-91dc-57ebff4da99e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.909891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41183323-e4cb-48c2-b26b-38b0a2b5a602 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:12.927721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:12.928830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:12.929284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:12.929770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:13.014529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:13.015763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:13.016258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:13.016795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:13.018383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=43338d760135414e9fb6716858b0858f) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:13.019083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:13.022784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:13.030917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:13.032748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:13.034367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 98/205] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/role_assignments?user.id=43338d760135414e9fb6716858b0858f&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 259 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:13.123425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:13.174667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 108/206] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:13.184818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:13.355347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0deb37-6524-41e0-95d9-f207e16035a0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:13.382463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:13.409135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:13.409811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:13.410090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:13.410360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:13.411784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:13.413272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b3ee0-14cb-4f63-9f8d-0fb30f8840f8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:13.415223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 99/207] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:13 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:13.424769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:13.563494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564c03f-14ac-4be8-862c-ba06241fd89d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:13.624522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:13.625572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:13.626099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:13.626574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:13.628029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f, user.id=c185b65860914a77acb771ce1045843c) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:13.628575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:13.631535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121ed5-c324-4c7c-a01b-a0a19f153fb7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:13.651425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 109/208] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:13 2022] GET /identity/v3/role_assignments?user.id=c185b65860914a77acb771ce1045843c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:14.029530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9908-94cf-480f-9b61-db54c7d5db83 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:14.030310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9908-94cf-480f-9b61-db54c7d5db83 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:14.030310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9908-94cf-480f-9b61-db54c7d5db83 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:14.031645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 100/209] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:14.038161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89bd13-c2d7-4897-9c1a-a00c661f8599 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:14.038550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89bd13-c2d7-4897-9c1a-a00c661f8599 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:14.038700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89bd13-c2d7-4897-9c1a-a00c661f8599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:14.059502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1e89bd13-c2d7-4897-9c1a-a00c661f8599 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:14.060720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1e89bd13-c2d7-4897-9c1a-a00c661f8599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:14.094785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:14.142900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 110/210] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:14.438674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc9908-94cf-480f-9b61-db54c7d5db83 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:14.467384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec935123-ded9-4530-8280-9825aab3cfbb None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:14.467913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec935123-ded9-4530-8280-9825aab3cfbb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:14.468082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec935123-ded9-4530-8280-9825aab3cfbb None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:14.469615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 111/211] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:14.479542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:14.480788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:14.481293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:14.514079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:14.515711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:14.631098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc9908-94cf-480f-9b61-db54c7d5db83 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:14.640153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 112/212] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:14.703282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:14.704348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:14.704844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:14.705725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:14.731115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:14.743410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:14.744356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:14.745155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:14.750091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 101/213] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:14.756284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:14.930099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:14.958555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:14.976313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14566228-43cb-4bd9-877d-2bcc26990a4e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:15.010045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:15.055684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:15.056768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.057799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.059212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.060181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] RBAC: Adding query filter params (name=member) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:15.060885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:15.064704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:15.083393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187c4e-65ba-4631-bc52-b58d1d41fabe None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:15.084875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 113/214] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:15.093865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:15.129178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:15.130368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.130872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.131364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.133314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:15.135160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:15.137070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 102/215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:15.143099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:15.274051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec073b7-2e0d-4f0d-883a-17f4dd25d3d0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:15.282109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbf4e1-3b94-4b09-b848-eeed3404aa83 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:15.295424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:15.319309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:15.320414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.320911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.321379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.322447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:15.324171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:15.361282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:15.361282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.361282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.361282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] PATH_INFO: `/v3/users/bedb357744544baf830f74deafe69d93` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.362340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 103/216] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:15 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:15.387623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] RBAC: Authorizing `identity:get_user(user_id=bedb357744544baf830f74deafe69d93)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:15.390654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c8ef-f262-423d-942e-8f57bb539f06 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:15.394301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 114/217] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:15 2022] GET /identity/v3/users/bedb357744544baf830f74deafe69d93 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:15.404794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:15.542659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185a0f9-50fc-4890-a0e6-a72bf3898aa5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:15.555522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604ead-e244-4235-ac64-fc027e16506a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.555522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604ead-e244-4235-ac64-fc027e16506a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604ead-e244-4235-ac64-fc027e16506a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.557798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 115/218] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:15.565522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f72da-cf98-4363-ad97-b991e9d80a06 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.566014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f72da-cf98-4363-ad97-b991e9d80a06 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.566493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f72da-cf98-4363-ad97-b991e9d80a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.598426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:15.600394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.600901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.601432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.602739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b49f72da-cf98-4363-ad97-b991e9d80a06 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:15.604824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b49f72da-cf98-4363-ad97-b991e9d80a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:15.627603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:15.670611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:15.711026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:15.713074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:15.731816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 116/219] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:15.733404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d234b9-58b9-4939-a850-640850aa5f3e None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.734996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d234b9-58b9-4939-a850-640850aa5f3e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.734996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d234b9-58b9-4939-a850-640850aa5f3e None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.736327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 117/220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:15.745075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:15.745426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:15.745853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:15.772378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:15.774622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:15.856765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 104/221] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:15.862706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:15.912246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 118/222] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:15.964749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:16.030599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77c56-d449-4494-ba99-0aed1e548fbb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:16.049028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.097123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:16.098369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:16.099198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:16.099629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:16.101515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:16.104424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:16.108162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 105/223] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:15 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:16.115858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.294884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:16.358934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0603-1a08-495f-81a5-61a2e240bf93 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:16.419749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5de229-2ae2-4429-9f21-8376fe41855b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:16.445505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:16.447089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:16.447877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:16.448623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:16.474790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:16.487549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:16.489313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:16.494794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:16.500134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 119/224] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:16.506852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.521852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:16.523798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:16.524253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:16.524693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:16.526393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=bedb357744544baf830f74deafe69d93) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:16.526871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:16.530313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:16.545459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 106/225] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/role_assignments?user.id=bedb357744544baf830f74deafe69d93&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=0da2f51a570245b1acf832a45de984cc => generated 259 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:16.547073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.564018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:16.797684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7ef8-f165-4f23-84a7-77f162878105 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:16.810386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c508eb2-03d5-42cb-86b1-0f867d313fc2 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:16.858000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:16.858759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:16.859102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:16.859383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:16.873511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:16.879648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:16.884092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:16.892808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 107/226] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:16.895457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:16.895457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:16.895457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:16.895457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:16.902052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:16.902052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:16.902560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.902823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:16.902823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:16.911392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 120/227] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:16.919562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:16.970694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:17.062095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff2942-23b7-4de1-9661-24a357ff5107 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.141722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166e70d-5322-4d29-ba36-805abc46e983 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.153826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.154528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.154735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.155013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.155938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:17.156131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.157583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.173556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 108/228] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:17.175242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:17.228193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.229291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.229831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.230330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.239109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.242610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.246557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 121/229] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:16 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:17.260761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:17.327454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df6945-8174-48ab-9a39-709879ccd8e0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.356262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:17.369212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.370343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.370795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.371222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.372630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.374312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.376137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 109/230] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:17 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:17.382769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:17.450833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:17.508393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c92d1-9d8b-4d26-8ce6-b710f60c550c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.554048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.555235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.555738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.556268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.557866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.559603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.561929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 110/231] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:17 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:17.567131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4482-f899-48e0-aa15-52c44edca1d3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.568726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:17.641020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.642966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.643434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.643904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.647324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.650751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.656254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 122/232] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:17 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:17.668834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:17.741737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:17.820675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debe49-3cd5-4b79-9a99-73751818ee4a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:17.889677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:17.889677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:17.889677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:17.889677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/43338d760135414e9fb6716858b0858f/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:17.891459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, user_id=43338d760135414e9fb6716858b0858f, role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:17.893607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b3245-77ef-42ad-956e-d88df7aaaa3d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:17.921117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 111/233] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:17 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/43338d760135414e9fb6716858b0858f/roles/acebbc3107e1448599b7b2bd0096b755 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:17.939464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71018c-a3b1-42f2-9dc1-e517ee9aefd7 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:18.040209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:18.041429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:18.041961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:18.042463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/43338d760135414e9fb6716858b0858f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:18.046611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, user_id=43338d760135414e9fb6716858b0858f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:18.049673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea115-d629-4c4a-b9ca-e3d850483bbd None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:18.054480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:18.079566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 123/234] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:17 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/43338d760135414e9fb6716858b0858f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:18.339686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1988bf-4d41-47c1-b6fd-aa594c6f4e87 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:18.340213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1988bf-4d41-47c1-b6fd-aa594c6f4e87 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:18.340675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1988bf-4d41-47c1-b6fd-aa594c6f4e87 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:18.343821 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 112/235] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:18.350451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68269-2ad5-4473-890b-2feefabfbee5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:18.350961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68269-2ad5-4473-890b-2feefabfbee5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:18.351069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:18.351279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:18.351456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c68269-2ad5-4473-890b-2feefabfbee5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:18.351555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:18.354117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 124/236] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:18.367188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:18.367698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:18.368201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:18.377107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:18.378349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:18.414467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:18.416578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:18.583701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 113/237] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:18.602259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 125/238] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:19.018689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:19.090387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:19.182357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:19.209919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be83e0-175e-440f-aa7c-238c91d332d4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:19.285043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:19.293679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:19.294716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:19.295389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:19.298981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:19.302198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:19.306836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 114/239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:19.315700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:19.333635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54165e-950a-4b4f-94df-d3d5013e7f29 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:19.386649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:19.416403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:19.417501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:19.418013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:19.418495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:19.447251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:19.460175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:19.462053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:19.463686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:19.467261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 126/240] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:19.474899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:19.575374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b4451-c422-4892-a24b-7cab93c88e6a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:19.625455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:19.670437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:19.671485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:19.671967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:19.672434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:19.673292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:19.675061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:19.703625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a49d-09dd-4f39-82cb-3b2858dcdfb4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:19.740766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 115/241] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:19 2022] POST /identity/v3/users => generated 293 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:19.785561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:19.786817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:19.787334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:19.787843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:19.788791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:19.789319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:19.791085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2a2ab-1e4b-4fdc-b9f7-d452a293f22a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:19.806968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 127/242] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:19.812495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:19.964737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc709-52c1-4003-93c7-56a4fb30dc0a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:20.016365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:20.017039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.017317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.017591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] PATH_INFO: `/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.042179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff1ea-7505-4119-95ff-010a483101d9 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.042935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff1ea-7505-4119-95ff-010a483101d9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.043409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff1ea-7505-4119-95ff-010a483101d9 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.045433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 128/243] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:20.048361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] RBAC: Authorizing `identity:get_user(user_id=40b70b7cb2c440fbbf282f2e6ee8045b)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:20.051026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:20.051887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.052650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.053197 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.054155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 116/244] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:19 2022] GET /identity/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:20.060662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:20.077421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:20.078713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:20.135550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:20.168646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:20.238870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 129/245] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:20.275582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c91f03-94a2-4b52-b996-c34e89083761 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:20.342378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:20.343431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.343921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.344402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.489093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:20.492347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:20.560721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:20.582402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:20.707194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 117/246] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 11:35:20.713856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:20.791689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c64e56-26d9-47e8-8a7e-ec69b43cdd54 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:20.837875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:20.838533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.838811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.839085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.852870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-6b7c98a4-535b-486e-9e63-5dee18b96514 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:20.858824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:20.860587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 130/247] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:20.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000c69-6ec3-4c9a-9d90-39442af6c3f4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:20.867838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000c69-6ec3-4c9a-9d90-39442af6c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:20.867838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000c69-6ec3-4c9a-9d90-39442af6c3f4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:20.869290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 131/248] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:20.870111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29000c69-6ec3-4c9a-9d90-39442af6c3f4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:20.932641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffcb2b-8339-4043-8a6f-92eefbdc73ac None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:20.972116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:21.018580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:21.020539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.022258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.023852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.026922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:21.031188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b721ac-0b2d-42eb-a98a-e46413c77227 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:21.035813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 118/249] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:20 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:21.039953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.040799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.041555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.060615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:21.067541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:21.069169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:21.147065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29000c69-6ec3-4c9a-9d90-39442af6c3f4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:21.151991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 119/250] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:21.153158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:21.195375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:21.196005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.196283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.196555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.197242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] RBAC: Adding query filter params (name=member) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:21.197546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:21.199004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:21.209298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 132/251] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:21.215047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:21.275730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549444-3452-40bd-b70c-8bb4999c5136 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:21.319658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:21.320693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:21.321558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.322139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.322697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.323694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=40b70b7cb2c440fbbf282f2e6ee8045b) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:21.324257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:21.325985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:21.334110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 120/252] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:21 2022] GET /identity/v3/role_assignments?user.id=40b70b7cb2c440fbbf282f2e6ee8045b&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:21.387733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:21.393257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faa08f-26ea-4d8c-871a-61d876e4e727 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:21.435016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:21.436037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.436490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.436930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] PATH_INFO: `/v3/users/bedb357744544baf830f74deafe69d93` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.438444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] RBAC: Authorizing `identity:get_user(user_id=bedb357744544baf830f74deafe69d93)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:21.440225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2855f3-7d40-4d15-85d0-4f9be795eea9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:21.442177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 133/253] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:21 2022] GET /identity/v3/users/bedb357744544baf830f74deafe69d93 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:21.459924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:21.622841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf30f9d-c1f8-4d5e-820b-e814ea10ee5f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:21.651244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1b173-88d9-4003-8523-39548d7411ee None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.651979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1b173-88d9-4003-8523-39548d7411ee None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.652427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1b173-88d9-4003-8523-39548d7411ee None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.654172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 134/254] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:21.663051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.663524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.664017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.681030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:21.686107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:21.686674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:21.687110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:21.695369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:21.696075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:21.697610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:21.698304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:21.702571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 121/255] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:21 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:21.708968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:21.864398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 135/256] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:21.894496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:21.973154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942f23-bf82-4466-a2a0-d987db561656 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:22.045788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:22.046897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:22.047355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:22.047822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/bedb357744544baf830f74deafe69d93/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:22.051086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, user_id=bedb357744544baf830f74deafe69d93, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:22.053223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:22.083939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 122/257] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:21 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/bedb357744544baf830f74deafe69d93/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:35:22.153345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:22.345848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:22.357227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6caa2-46b6-49bd-b198-bcb1e6e319c8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:22.412270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:22.413207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:22.413508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:22.413742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:22.432492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:22.438520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:22.440154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:22.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 136/258] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:22.449882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:22.614013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:22.702386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae3cc-d35f-41e2-a503-83af85feb1b1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:22.764829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:22.813680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:22.813680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:22.813680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:22.813680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:22.814095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:22.814095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:22.814892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:22.839297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 123/259] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:22.854790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:22.888067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b792910-c40c-42e0-92ef-e8bf513a0bc5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:22.991664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:22.992235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:22.992512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:22.992777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:23.009970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:23.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:23.018427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:23.019638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 137/260] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:23.024545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:23.050543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:23.131923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798edc15-33c5-41d3-a716-55c00f564820 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:23.189059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba7a9-713d-421f-b669-e0c1e56e581e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:23.201151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:23.201848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:23.202126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:23.202399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:23.203614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:23.205109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:23.206983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 124/261] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:22 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:23.218741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:23.260788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:23.261026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:23.261865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:23.262454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:23.265688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:23.265688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:23.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:23.284206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 138/262] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:23.294674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:23.320219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:23.447749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8ad00-0c6b-40fb-af78-480b94aadd17 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:23.490565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:23.509197 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ec890-33a8-45bc-a843-a4e9b37bfbf8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:23.558871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:23.559510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:23.559783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:23.560052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:23.574791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:23.575866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:23.576523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:23.576999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] PATH_INFO: `/v3/users/43338d760135414e9fb6716858b0858f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:23.587767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=43338d760135414e9fb6716858b0858f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:23.590917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:23.604386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 139/263] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:23 2022] GET /identity/v3/users/43338d760135414e9fb6716858b0858f => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:23.617661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:23.817197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:23.819622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:23.885253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aa5a8-f5f9-44d1-a8d8-aa1daa104bf2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:23.941900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:23.977379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:23.978867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:23.979683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:23.980478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.022059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:24.081002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 125/264] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 869 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 11:35:24.090788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:24.125601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:24.128719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a940dff-3639-4573-90c7-3691ab6c38f3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:24.326845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3f5b-89d7-490f-ae2b-a61fdee7afcf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:24.339210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 140/265] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:24.341300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325682b2-f6ca-484d-8c45-58b067ec800b None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.341842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325682b2-f6ca-484d-8c45-58b067ec800b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.342333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325682b2-f6ca-484d-8c45-58b067ec800b None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.345128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 141/266] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:24.347803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a06-136a-4caa-8d17-f6dd5948a181 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.348325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a06-136a-4caa-8d17-f6dd5948a181 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.348873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a06-136a-4caa-8d17-f6dd5948a181 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.356759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 142/267] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:24.356759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52a06-136a-4caa-8d17-f6dd5948a181 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:24.377231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:24.378427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.378927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.379400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.380956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:24.382966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f15b4-f0c3-48b8-b3cd-b4004e85d966 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:24.385461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 126/268] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:24 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:24.387037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d3e40-69ac-41ff-a417-4da24a7f8ea6 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.387547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d3e40-69ac-41ff-a417-4da24a7f8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.388020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d3e40-69ac-41ff-a417-4da24a7f8ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.409039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-384d3e40-69ac-41ff-a417-4da24a7f8ea6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:24.410865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-384d3e40-69ac-41ff-a417-4da24a7f8ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:24.431664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:24.454124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:24.505707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 127/269] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:24.505707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268577-c525-4548-a224-3ecfc434cabb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.505707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268577-c525-4548-a224-3ecfc434cabb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.505707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268577-c525-4548-a224-3ecfc434cabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.546561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52a06-136a-4caa-8d17-f6dd5948a181 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:24.548641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c2268577-c525-4548-a224-3ecfc434cabb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:24.550976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c2268577-c525-4548-a224-3ecfc434cabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:24.596440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:24.597160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.597511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.597838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.600778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:24.602649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:24.605851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 143/270] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:24 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:24.607381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:24.682567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 128/271] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:24.684373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8328-6f6a-4885-818d-01c7fde7e85a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.684869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8328-6f6a-4885-818d-01c7fde7e85a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.685375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8328-6f6a-4885-818d-01c7fde7e85a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.687409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 129/272] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:24.688477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8328-6f6a-4885-818d-01c7fde7e85a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:24.862130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae80b5-9cb7-4273-b33b-cddfc39c7941 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:24.883923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:24.914857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:24.915993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.916476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.916948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.917962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=43338d760135414e9fb6716858b0858f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:24.918505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:24.920243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724db9c4-b74c-4962-8466-1893bbc5e618 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:24.928487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 144/273] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:24 2022] GET /identity/v3/role_assignments?user.id=43338d760135414e9fb6716858b0858f&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:24.930846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c49a3-04e1-43aa-ad13-76dd646ae999 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:24.931393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c49a3-04e1-43aa-ad13-76dd646ae999 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:24.932008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c49a3-04e1-43aa-ad13-76dd646ae999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:24.941143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:24.961765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-476c49a3-04e1-43aa-ad13-76dd646ae999 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:24.964006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-476c49a3-04e1-43aa-ad13-76dd646ae999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:24.999360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8328-6f6a-4885-818d-01c7fde7e85a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.061780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.062450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.062732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.063128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.063910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=43338d760135414e9fb6716858b0858f) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:25.064221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:25.066301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:25.081095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 130/274] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:24 2022] GET /identity/v3/role_assignments?user.id=43338d760135414e9fb6716858b0858f&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 608 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:25.082826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.099965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 145/275] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:25.101325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8194ef-e187-400d-a612-d8eaf0708e72 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.101730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8194ef-e187-400d-a612-d8eaf0708e72 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.102011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8194ef-e187-400d-a612-d8eaf0708e72 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.103790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 146/276] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:25.109397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.109800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.110082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.134023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:25.135191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:25.220857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 147/277] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:25.222195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.272822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:25.352621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480078a-f8f6-40fc-9dc2-027c1604cb33 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.371535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837d74-7bdb-49f5-bb0a-88b42480f695 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.412100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.413122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.413495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.413906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.429520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:25.429858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.430770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.431186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.431602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:25.435684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:25.436630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:25.437417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 148/278] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:25.443500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.452372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:25.461002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:25.462035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:25.462853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:25.463445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 131/279] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:25.464123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.477497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:25.628488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:25.636598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff266a5d-aa3a-42f1-821d-3abe894c5ef2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.639702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc9de8-5bdf-4e73-a1f8-ab507562d5bd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.695145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.696141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.696597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.697047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.698487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:25.698981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:25.699380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.700020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.700288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.700639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.701678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:25.701991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:25.703484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:25.711248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 132/280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:25.711997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.718813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 149/281] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:25.720683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.819507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:25.853061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29706781-15a2-41e9-acd7-bf87151a211d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.867783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c83d7-c988-4ca5-b351-bee3e4034cc4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:25.895307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.896513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.896927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.897327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.898173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:25.898582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:25.900279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:25.908746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:25.909379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:25.909682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:25.909964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:25.910393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 133/282] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:25.910659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:25.911533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.912264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:25.942184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 150/283] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:25 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:25.944004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:25.979629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:26.119409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72bc0b-9227-4bbb-9b42-5a9301ecc281 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.121094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9473-16b6-489e-9eee-2115b0328324 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.176797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.177737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.178122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.178556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] PATH_INFO: `/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.180289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] RBAC: Authorizing `identity:get_user(user_id=40b70b7cb2c440fbbf282f2e6ee8045b)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:26.182161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:26.191520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 134/284] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:26.191520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.193429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:26.193685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af501-992c-43b2-958b-694337d4733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.195068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af501-992c-43b2-958b-694337d4733d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.195415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af501-992c-43b2-958b-694337d4733d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.195833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af501-992c-43b2-958b-694337d4733d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.211521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137af501-992c-43b2-958b-694337d4733d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-137af501-992c-43b2-958b-694337d4733d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:26.217703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:26.218667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 151/285] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:26.219439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af501-992c-43b2-958b-694337d4733d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.354176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:26.383296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a595f-07fc-4981-83ab-e1ced0be0857 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.398048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af501-992c-43b2-958b-694337d4733d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.432992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.433766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.433948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.434219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] PATH_INFO: `/v3/users/6a282d2b293d4242b8365c18702e4565` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.453715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] RBAC: Authorizing `identity:get_user(user_id=6a282d2b293d4242b8365c18702e4565)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:26.455596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:26.457364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 135/286] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:26 2022] GET /identity/v3/users/6a282d2b293d4242b8365c18702e4565 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:26.463103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.473188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.475483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.476061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.476541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.478610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:26.480539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:26.482927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 152/287] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:26 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:26.484211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.538694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:26.592260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45275d41-fbf8-4d99-b23a-7f6dff1f9e7c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.614442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574e18-7de2-4c6c-922b-f2b1c1e7f4fc None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.663510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.664314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:26.665424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.665957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.666394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.666542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:26.667524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:26.667691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:26.668136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:26.668731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:26.670561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, user.id=6a282d2b293d4242b8365c18702e4565, scope.system=all) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:26.670772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:26.671623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:26.675261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:26.690810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 153/288] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 6 11:35:26 2022] GET /identity/v3/role_assignments?user.id=6a282d2b293d4242b8365c18702e4565&scope.system=all&role.id=0da2f51a570245b1acf832a45de984cc => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:26.692814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.703704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 136/289] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:26.706148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:26.733339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:26.954987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fad58-dbdd-4aa8-b936-8137a7c9135a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:26.960732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f223d9-9ed8-4b40-a8f1-6049009c7994 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:27.013530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:27.033782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:27.034974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:27.035255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:27.035529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/40b70b7cb2c440fbbf282f2e6ee8045b/roles/5af87c99a4b241e7b310b566fca557c5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:27.038967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, user_id=40b70b7cb2c440fbbf282f2e6ee8045b, role_id=5af87c99a4b241e7b310b566fca557c5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:27.040453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:27.042598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:27.043126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:27.043397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:27.043717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] PATH_INFO: `/v3/users/bedb357744544baf830f74deafe69d93` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:27.046157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] RBAC: Authorizing `identity:get_user(user_id=bedb357744544baf830f74deafe69d93)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:27.047801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:27.051304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 137/290] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:26 2022] GET /identity/v3/users/bedb357744544baf830f74deafe69d93 => generated 297 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:27.060296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:27.073152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 154/291] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:35:26 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/users/40b70b7cb2c440fbbf282f2e6ee8045b/roles/5af87c99a4b241e7b310b566fca557c5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:27.236045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccb9a-b84a-4933-99a9-677a0cbe6a44 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:27.306246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:27.306958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:27.307236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:27.307622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:27.400710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:27.417711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:27.419981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:27.575223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 138/292] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:27.584235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:27.737559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:27.803410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d722b-2c7d-4e78-af54-febfa9dbc6c4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:27.872661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:27.873246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:27.873526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:27.873831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:27.875083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:27.876600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ff27c-a526-48ee-80c6-ee7b14b74f21 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:27.878774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 155/293] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:27 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:27.888210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:28.020006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:28.032020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55820b44-1cc8-40ab-b8be-1361cb28e492 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:28.079126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:28.079745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:28.080018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:28.080284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:28.081029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=bedb357744544baf830f74deafe69d93) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:28.081331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:28.082853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bd34e-3ea5-48eb-a8be-98dcd8ddbdec None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:28.095051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 139/294] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:27 2022] GET /identity/v3/role_assignments?user.id=bedb357744544baf830f74deafe69d93&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=0da2f51a570245b1acf832a45de984cc => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:29.153350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04aad4-9169-4e6d-8f66-9ce2583e1f47 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.154125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04aad4-9169-4e6d-8f66-9ce2583e1f47 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.154125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04aad4-9169-4e6d-8f66-9ce2583e1f47 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.155673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 156/295] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:29.163544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6172c07-442c-4edf-bdf6-bbf6cc99b987 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.164765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6172c07-442c-4edf-bdf6-bbf6cc99b987 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.165250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6172c07-442c-4edf-bdf6-bbf6cc99b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.187692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a6172c07-442c-4edf-bdf6-bbf6cc99b987 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:29.189427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a6172c07-442c-4edf-bdf6-bbf6cc99b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:29.273398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 140/296] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:29.327424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472b3d-75fb-4618-9095-5a7581dc1411 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.327942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472b3d-75fb-4618-9095-5a7581dc1411 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.328383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12472b3d-75fb-4618-9095-5a7581dc1411 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.329979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 157/297] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:29.336799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ffd63-3f96-4761-94b5-9ce6f420dfa1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.337367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ffd63-3f96-4761-94b5-9ce6f420dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.337862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ffd63-3f96-4761-94b5-9ce6f420dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.358043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ad3ffd63-3f96-4761-94b5-9ce6f420dfa1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:29.359586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ad3ffd63-3f96-4761-94b5-9ce6f420dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:29.443696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 141/298] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:29.562761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12472b3d-75fb-4618-9095-5a7581dc1411 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:29.653070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e46-4416-4ab3-9022-95d06b2519b0 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.653360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e46-4416-4ab3-9022-95d06b2519b0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.653688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e46-4416-4ab3-9022-95d06b2519b0 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.655084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 142/299] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:29.661736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.662380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.662873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.689009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:29.691340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:29.697851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12472b3d-75fb-4618-9095-5a7581dc1411 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:29.708872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:29.745828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:29.746481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.746767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.747025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.752868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:29.762829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:29.768685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:29.769987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 158/300] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:29.771434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:29.798786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 143/301] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:29.800102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:29.908506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99399b0-7aae-486e-8295-95e7e5776433 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:29.946708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64cb63-77c7-42af-9dba-fdaade690b02 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:29.977602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:29.978426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.978859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.979585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.981046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:29.983394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0438f-0709-4d5e-a628-92bf7c7c1939 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:29.986664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 159/302] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:29.988566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a4ac-14c8-4126-b891-710c1ac38bbe None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:29.989049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a4ac-14c8-4126-b891-710c1ac38bbe None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:29.989529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a4ac-14c8-4126-b891-710c1ac38bbe None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:29.993702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 160/303] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:29.993990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a4ac-14c8-4126-b891-710c1ac38bbe None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.004461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.005902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.006569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.007101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.008208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:30.008662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:30.010261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda75082-bc92-4f57-98e8-c00d9e1f405e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:30.021117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 144/304] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:30.023155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.023440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.023711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.045933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:30.047373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:30.050091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:30.130136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:30.173684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 145/305] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:30.173684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.202278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a4ac-14c8-4126-b891-710c1ac38bbe None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.269248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.270340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.270822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.271796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] PATH_INFO: `/v3/groups` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.273009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:30.275951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:30.300831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 161/306] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 6 11:35:29 2022] POST /identity/v3/groups => generated 229 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:30.302792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.381131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf9483-0aab-4c9d-865d-b7edb7fead60 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.468043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.469538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.470286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.470948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] PATH_INFO: `/v3/users/b1df7f7bff5e4822bde57f3cdd879e22` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.488451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:30.511535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=b1df7f7bff5e4822bde57f3cdd879e22)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:30.516565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:30.516953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a808ec9-c50c-41b4-8548-bfe5efd1a8ee None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.517983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 146/307] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/users/b1df7f7bff5e4822bde57f3cdd879e22 => generated 293 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:30.524917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.553348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:30.577626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.578767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.579273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.579840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.601627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:30.612815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:30.613830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:30.615055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:30.621469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 162/308] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:30.621469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.675593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38422f7-67e6-469c-a2bb-5c2ed2237eb1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.720301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.720953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.721233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.721483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.734791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:30.740164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:30.741962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 147/309] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:30.743531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.828038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:30.837876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109b7eb-25b3-4b03-8d4e-ff090b5a2bd6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.887141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.888192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.888667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.889144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.890565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, user.id=b1df7f7bff5e4822bde57f3cdd879e22, scope.system=all) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:30.891122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:30.893878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:30.903472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 163/310] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/role_assignments?user.id=b1df7f7bff5e4822bde57f3cdd879e22&scope.system=all&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:30.904337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:30.919903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062510b-35e7-4574-b6f6-edac80e7a515 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:30.988028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:30.988714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:30.989007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:30.989254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:30.990740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:30.991037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:30.992501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:30.993696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:31.005692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 148/311] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:31.014995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:31.126034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ced67-e2b7-47d3-9bea-583acbd4f21e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:31.189932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:31.190747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:31.191121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:31.191487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:31.192302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:31.192704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:31.194927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:31.205293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 164/312] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:31.211107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:31.238963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf07ea1-b9b8-40e2-b6c1-490a09936849 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:31.286173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:31.310552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:31.311110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:31.311395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:31.311688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] PATH_INFO: `/v3/users/6a282d2b293d4242b8365c18702e4565` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:31.313379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] RBAC: Authorizing `identity:get_user(user_id=6a282d2b293d4242b8365c18702e4565)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:31.316071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:31.319070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 149/313] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:31 2022] GET /identity/v3/users/6a282d2b293d4242b8365c18702e4565 => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:31.328512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:31.404678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b0bc5-49e7-461f-b1ab-708b10cf8761 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:31.429587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:31.476516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:31.477571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:31.478068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:31.478501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] PATH_INFO: `/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:31.480555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] RBAC: Authorizing `identity:get_user(user_id=40b70b7cb2c440fbbf282f2e6ee8045b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:31.483425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:31.486794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 165/314] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:31 2022] GET /identity/v3/users/40b70b7cb2c440fbbf282f2e6ee8045b => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:31.495308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:31.574429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f8f48-8e86-4b47-bc6c-dac9d0efc75c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:31.641387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:31.642588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:31.643059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:31.643548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] PATH_INFO: `/v3/system/users/6a282d2b293d4242b8365c18702e4565/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:31.646984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a282d2b293d4242b8365c18702e4565, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:31.649489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:31.677896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 150/315] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 6 11:35:31 2022] PUT /identity/v3/system/users/6a282d2b293d4242b8365c18702e4565/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:35:31.710022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747f81-eec7-472f-8eef-79ea9612ce51 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:31.787212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:31.788297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:31.788752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:31.789191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:31.886455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:31.934050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:31.936492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:32.140105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 166/316] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:32.147457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:32.219039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:32.345155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1a756-a598-4ca4-8927-4215c5fe38c1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:32.410842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:32.412495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:32.413313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:32.414115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:32.416696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:32.420082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663d0e0-3175-468e-b6f8-a02a42621f3f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:32.424247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 151/317] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:32 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:32.430669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:32.452175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678d99-425d-4f75-acc2-f4e3e0dc8be6 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:32.452519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678d99-425d-4f75-acc2-f4e3e0dc8be6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:32.452726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678d99-425d-4f75-acc2-f4e3e0dc8be6 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:32.454124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 152/318] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:32.460526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:32.460811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:32.461077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:32.481410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:32.482617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:32.550857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:32.574434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 153/319] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:32.589868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d5462-318d-490b-8fac-5e61a2d1d9f6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:32.631739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:32.632370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:32.632649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:32.633166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:32.633947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13, user.id=40b70b7cb2c440fbbf282f2e6ee8045b) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:32.634256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:32.635715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:32.642979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 167/320] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:35:32 2022] GET /identity/v3/role_assignments?user.id=40b70b7cb2c440fbbf282f2e6ee8045b&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:32.859881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:32.920025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:33.008706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f33af-3c6e-462d-b82d-7b764e53ba2b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:33.050169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:33.050833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.051107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.051376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:33.053337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:33.054933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8755044-8a44-4e04-8024-1d3791604e2d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:33.057018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 154/321] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:33.063093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:33.217900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670df6cb-7273-4cba-bfd9-401dfdd36d20 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:33.286150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:33.287300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.287764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.288337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] PATH_INFO: `/v3/groups` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:33.289565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:33.292508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be24c02-bb72-4755-8eb3-cab5be9da3d0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:33.329693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 168/322] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 6 11:35:33 2022] POST /identity/v3/groups => generated 226 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:33.516560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b8c5-e99b-4797-a7b4-3f980f2cf22e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.517227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b8c5-e99b-4797-a7b4-3f980f2cf22e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.517367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b8c5-e99b-4797-a7b4-3f980f2cf22e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:33.519926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 155/323] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:33.530171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42815c2-4746-4298-9179-a54a3714772d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.530751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42815c2-4746-4298-9179-a54a3714772d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.531108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42815c2-4746-4298-9179-a54a3714772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:33.542113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:33.560341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e42815c2-4746-4298-9179-a54a3714772d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:33.561940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e42815c2-4746-4298-9179-a54a3714772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:33.675745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 169/324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:33.964765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71b8c5-e99b-4797-a7b4-3f980f2cf22e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:33.975372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49196-b9c0-456f-a7a6-9b19fb17c1c6 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.976277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49196-b9c0-456f-a7a6-9b19fb17c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.976967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49196-b9c0-456f-a7a6-9b19fb17c1c6 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:33.979834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 170/325] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:33.993014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e316b-5cae-4055-921e-64f98914b783 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:33.994008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e316b-5cae-4055-921e-64f98914b783 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:33.994814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e316b-5cae-4055-921e-64f98914b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:34.035467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-973e316b-5cae-4055-921e-64f98914b783 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:34.038284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-973e316b-5cae-4055-921e-64f98914b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:34.172498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 171/326] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:34.216735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71b8c5-e99b-4797-a7b4-3f980f2cf22e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:34.259946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:34.292651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:34.293813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:34.294318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:34.294812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:34.318830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:34.329057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:34.331037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:34.332842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:34.337284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 156/327] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:34.343661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e316b-5cae-4055-921e-64f98914b783 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:34.485790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:34.529486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:34.633389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e316b-5cae-4055-921e-64f98914b783 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:34.737465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:34.738022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:34.738303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:34.738581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:34.739276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:34.739578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:34.742504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:34.765006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 172/328] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:34.772228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:34.818238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57211ebd-216c-4c31-8c41-781045d53b72 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:34.918247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:34.919837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:34.920632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:34.921405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:34.955625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:34.969794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:34.971642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:34.974830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:34.982647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 157/329] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:34.990873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:35.059226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba615380-1b38-4e00-a2ff-20c8a10e3678 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:35.078205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:35.116839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:35.143851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:35.144877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:35.145344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:35.146086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] PATH_INFO: `/v3/users/b1df7f7bff5e4822bde57f3cdd879e22` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:35.150214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] RBAC: Authorizing `identity:get_user(user_id=b1df7f7bff5e4822bde57f3cdd879e22)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:35.152879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:35.159128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 173/330] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:34 2022] GET /identity/v3/users/b1df7f7bff5e4822bde57f3cdd879e22 => generated 293 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:35.169728 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:35.308209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f795-a6eb-471f-a691-debb36e39323 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:35.401683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:35.401683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:35.401683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:35.402034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:35.405677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:35.405677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:35.405677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:35.444487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 158/331] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:35.453753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:35.536653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8260be-bcf7-4ead-bfac-a4fa5b10dc1e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:35.633996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104140-92a6-4d24-9787-017499ed723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:35.635082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104140-92a6-4d24-9787-017499ed723d None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:35.635564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104140-92a6-4d24-9787-017499ed723d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:35.636005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104140-92a6-4d24-9787-017499ed723d None admin] PATH_INFO: `/v3/system/users/b1df7f7bff5e4822bde57f3cdd879e22/roles/5af87c99a4b241e7b310b566fca557c5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:35.643104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85104140-92a6-4d24-9787-017499ed723d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b1df7f7bff5e4822bde57f3cdd879e22, role_id=5af87c99a4b241e7b310b566fca557c5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:35.645594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85104140-92a6-4d24-9787-017499ed723d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:35.670979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 174/332] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 6 11:35:35 2022] PUT /identity/v3/system/users/b1df7f7bff5e4822bde57f3cdd879e22/roles/5af87c99a4b241e7b310b566fca557c5 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:35.781443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:35.791449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4eba7-f34e-40a9-90c2-69d51e91c67a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:35.844715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:35.845382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:35.845692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:35.845946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] PATH_INFO: `/v3/users/6a282d2b293d4242b8365c18702e4565` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:35.847342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] RBAC: Authorizing `identity:get_user(user_id=6a282d2b293d4242b8365c18702e4565)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:35.848850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3140-1c19-4671-b9dc-9603c75596da None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:35.851482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 159/333] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:35 2022] GET /identity/v3/users/6a282d2b293d4242b8365c18702e4565 => generated 293 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:35.858424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104140-92a6-4d24-9787-017499ed723d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:35.961410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:36.051026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104140-92a6-4d24-9787-017499ed723d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:36.121947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:36.122601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:36.122891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:36.123179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:36.124290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] RBAC: Adding query filter params (role.id=0da2f51a570245b1acf832a45de984cc, user.id=6a282d2b293d4242b8365c18702e4565, scope.system=all) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:36.124807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:36.127553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2916-7f52-437b-9915-5da85a26e4be None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:36.149085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 175/334] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 6 11:35:35 2022] GET /identity/v3/role_assignments?user.id=6a282d2b293d4242b8365c18702e4565&scope.system=all&role.id=0da2f51a570245b1acf832a45de984cc => generated 510 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:36.901041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffbaf7-890e-4437-8db6-5f88d5e9aec3 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:36.901711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffbaf7-890e-4437-8db6-5f88d5e9aec3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:36.902023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffbaf7-890e-4437-8db6-5f88d5e9aec3 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:36.905155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 160/335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:36.914077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e173-2e62-48d2-ae64-42e6710f020c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:36.914617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e173-2e62-48d2-ae64-42e6710f020c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:36.914617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e173-2e62-48d2-ae64-42e6710f020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:36.937589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-be41e173-2e62-48d2-ae64-42e6710f020c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:36.938896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-be41e173-2e62-48d2-ae64-42e6710f020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:37.031203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 176/336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:37.053085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079460c3-8075-4fc6-ab2a-6d049c1a26c9 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:37.053422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079460c3-8075-4fc6-ab2a-6d049c1a26c9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:37.053830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079460c3-8075-4fc6-ab2a-6d049c1a26c9 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:37.055253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 161/337] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:37.063850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca788-3e49-472f-b368-c6166450dd43 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:37.064194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca788-3e49-472f-b368-c6166450dd43 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:37.064416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca788-3e49-472f-b368-c6166450dd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:37.092939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a9fca788-3e49-472f-b368-c6166450dd43 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:37.094275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a9fca788-3e49-472f-b368-c6166450dd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:37.153156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:37.204429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 177/338] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:37.737553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cc95-60b2-4291-b064-20701bbad8b3 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:37.738674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cc95-60b2-4291-b064-20701bbad8b3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:37.739465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cc95-60b2-4291-b064-20701bbad8b3 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:37.743468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 162/339] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:37.755087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:37.756113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:37.757060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:37.795939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:37.798599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:37.916985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 178/340] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:37.921389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61cc95-60b2-4291-b064-20701bbad8b3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.182441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.191552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61cc95-60b2-4291-b064-20701bbad8b3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:38.276138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:38.277235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:38.277738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:38.278234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:38.299454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:38.310486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:38.311519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:38.312358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:38.313055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 163/341] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:38.335743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.392908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:38.488955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd4f9f-c197-4314-bb3f-09a113707371 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:38.506399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:38.569556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:38.570202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:38.570480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:38.570734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:38.581708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39148e7e-2870-4ac4-9094-cb3d7f89ceb7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:38.598186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:38.606506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:38.608725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 179/342] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:38.610601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.656725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:38.657367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:38.657664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:38.657971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:38.658680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:38.658988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:38.660448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:38.685362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 164/343] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:38.686576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.810599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b7e43-4ea4-4ca5-8d85-71e64a9a3f75 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:38.861743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:38.862476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:38.862749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:38.863018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:38.890254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:38.898398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:38.900872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 180/344] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:38.904309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:38.919233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:38.944896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:38.950120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b776f5-3f24-4fca-812c-67f2a47c0e4a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:39.027790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:39.029519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.031848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.033940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.035537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:39.037211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:39.042651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:39.080275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 165/345] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:39.081783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:39.169552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a3dfc-98dc-4377-a7c9-a37a32bdeab6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:39.245477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:39.246065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.246346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.246618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.317522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeea00-b7d1-45fe-b922-cb6d09659060 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:39.375034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:39.375585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.375913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.376200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.376892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:39.377193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:39.378714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:39.402264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 166/346] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:39.403653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:39.405356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:39.407084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00eaeb9-7f48-4c84-b1f9-187542876b8c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:39.434792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:39.446046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:39.538044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 181/347] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:39.539469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638bd4-3ebb-4c6b-af33-9d1264a6061a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.539780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638bd4-3ebb-4c6b-af33-9d1264a6061a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.540076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638bd4-3ebb-4c6b-af33-9d1264a6061a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.541617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 182/348] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:39.542497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638bd4-3ebb-4c6b-af33-9d1264a6061a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:39.611082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319d4e-9c69-4cd3-a9f9-b178531631f4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:39.679872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:39.680537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638bd4-3ebb-4c6b-af33-9d1264a6061a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:39.680811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.681092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.681364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.720577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:39.740377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:39.741888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:39.742019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:39.743516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:39.779783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:39.781326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:39.800387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:39.848259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:39.851118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce94d00-f2e9-4bbb-9ec7-18cf29b9bb1b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:39.967423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 167/349] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:39.968779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.036903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 183/350] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:40.038423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f415d8e-22ba-4a05-be2d-67e108ed3055 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.038720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f415d8e-22ba-4a05-be2d-67e108ed3055 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.038992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f415d8e-22ba-4a05-be2d-67e108ed3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.063413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3f415d8e-22ba-4a05-be2d-67e108ed3055 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:40.066489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3f415d8e-22ba-4a05-be2d-67e108ed3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:40.104449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:40.174932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc7aa6-ad6b-4bf6-b144-c1d027a317ca None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:40.200651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 184/351] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:40.201824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:40.204463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df34-954d-44fb-9a7f-3ce6a7490295 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.204952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df34-954d-44fb-9a7f-3ce6a7490295 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.205435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df34-954d-44fb-9a7f-3ce6a7490295 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.208012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 185/352] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:40.210187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5742977-8ab0-4115-85fd-44790cf1ef89 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.210649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5742977-8ab0-4115-85fd-44790cf1ef89 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.211122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5742977-8ab0-4115-85fd-44790cf1ef89 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.213894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 186/353] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:40.215086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5742977-8ab0-4115-85fd-44790cf1ef89 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.257379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:40.260652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.260652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.260652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.266086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:40.268782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:40.272250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 168/354] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:39 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:40.273557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.442488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5742977-8ab0-4115-85fd-44790cf1ef89 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:40.501952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c4c7b-da05-4c04-a608-f41a3b525faa None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:40.523409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:40.523409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.523900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.524507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.526920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:40.530111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5e44-6e1a-4653-af5d-56e2bc76f9d7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:40.533748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 187/355] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:40.536281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.537357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.537867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.561238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:40.562421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.562926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.563413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.564947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:40.566713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ab9a0-8d5f-4138-aef6-3ac3604177ed None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:40.569346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 169/356] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:40.571260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.571471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:40.571753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.572221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.573410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:40.602767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:40.604794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:40.606985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:40.693986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 188/357] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:40.694928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.699581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:40.743240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 170/358] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:40.744177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.887740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd6933-c368-4981-a9f3-d99a55f32aee None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:40.906333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e617849-35a4-4ab1-9b6b-dc0cc50d156e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:40.946817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a894122-25aa-430f-8257-616b04321119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:40.947500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a894122-25aa-430f-8257-616b04321119 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.947798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a894122-25aa-430f-8257-616b04321119 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.948105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a894122-25aa-430f-8257-616b04321119 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.958512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a894122-25aa-430f-8257-616b04321119 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:40.960947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a894122-25aa-430f-8257-616b04321119 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:40.963790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:40.963790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:40.963790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:40.963790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:40.964952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:40.965785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 171/359] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:40.966779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:40.967073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a894122-25aa-430f-8257-616b04321119 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:40.968979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 189/360] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:40.970066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.006072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.040269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.125630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2448360-4969-4b62-a90d-590c6bc1afa4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.145256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a894122-25aa-430f-8257-616b04321119 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.185688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.186186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.186467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.186745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.187978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:41.189458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:41.191514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 190/361] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:41.192358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.193995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.194673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.194964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.195283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.215624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:41.223703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:41.225338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:41.226184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:41.227358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 172/362] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:41.228463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.354984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.374261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c2ab3-beae-4b8d-8bf1-d31580e19bb3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.398387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d147bff-322d-470d-afb9-e352ab522d25 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.400749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.437169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.438323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.438822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.439309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.441111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:41.441317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:41.446841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.446841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.446841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.446841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.447254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:41.447254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:41.447788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:41.448408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:41.455959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 173/363] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=0da2f51a570245b1acf832a45de984cc => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:41.456928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.461988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 191/364] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:41.463558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.658080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2e26-07e6-483a-a6d1-1edf752fc46e None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.658451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a485c-2516-48c2-a7d3-884a06690e1d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.719809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.720536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.720647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.720892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.721304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.721772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.722236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:41.722357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.722636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:41.722755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.724716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:41.732936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 192/365] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=0da2f51a570245b1acf832a45de984cc => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:41.733948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.757974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.760166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:41.778982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:41.779961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 11:35:41.780774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:41.787391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 174/366] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:41.789031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:41.890610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae66a-25dd-4f5b-adef-9e17109e27df None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:41.913006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:41.970892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:41.972372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:41.973136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:41.973921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:41.976866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:41.977700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:41.980501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:42.003849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 193/367] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:42.006494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:42.118085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:42.145732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a731dd4-0279-4fe5-accc-2adae6579459 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:42.186766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d50a-7400-48fc-bc74-b9054180d881 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:42.188844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:42.189931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:42.190428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:42.190930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:42.191892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:42.192427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:42.194194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:42.204470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 194/368] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:42.205990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:42.265565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:42.266885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:42.267493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:42.268148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:42.306333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:42.331886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:42.332850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:42.334286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:42.336249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 175/369] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 548 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:42.339255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:42.466340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca462d-b14a-4024-8d60-473222413b17 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:42.516396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:42.542399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdebd5a-cfac-4e20-a8ab-9a28cca717b4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:42.550511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:42.551156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:42.551432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:42.551699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:42.593218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:42.671783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:42.672795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:42.673244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:42.673706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] PATH_INFO: `/v3/users/b1df7f7bff5e4822bde57f3cdd879e22` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:42.678893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] RBAC: Authorizing `identity:get_user(user_id=b1df7f7bff5e4822bde57f3cdd879e22)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:42.681372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:42.684245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 176/370] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:35:42 2022] GET /identity/v3/users/b1df7f7bff5e4822bde57f3cdd879e22 => generated 293 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:42.691408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:42.721037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:42.724635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:42.926385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 195/371] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:42.927936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:42.960760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e1f1-f38a-4f0a-930b-2256786293d7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:42.983225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:43.052755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fdcf1-ea97-4268-add2-6550e0711737 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:43.074434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:43.075078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:43.075362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:43.075637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:43.076333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:43.076633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:43.078112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:43.093714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:43.094884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:43.095322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:43.095697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:43.096561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] RBAC: Adding query filter params (role.id=5af87c99a4b241e7b310b566fca557c5, user.id=b1df7f7bff5e4822bde57f3cdd879e22, scope.system=all) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:43.096996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:43.098634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:43.104396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 177/372] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:43.105291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:43.115239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 196/373] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 6 11:35:42 2022] GET /identity/v3/role_assignments?user.id=b1df7f7bff5e4822bde57f3cdd879e22&scope.system=all&role.id=5af87c99a4b241e7b310b566fca557c5 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:43.116302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:43.194252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:43.315629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:43.317204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497ea5-4f70-4a9c-95d3-72b3664c5cd9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:43.385046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:43.385707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:43.386049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:43.386333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:43.491096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfadfb-cf34-4461-ba8f-c4fc92fbd1bb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:43.547628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:43.549022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:43.574892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:43.575552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:43.575832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:43.576103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:43.578266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:43.579759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:43.581471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 178/374] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:43 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:43.593854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:43.790056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:43.825723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1a990-6c85-4c3d-9b78-3c428b63c325 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:43.862082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:43.880400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 197/375] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:43.886921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:43.900524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:43.901620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:43.902128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:43.903458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:43.905813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:43.909635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:43.913821 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 179/376] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:43 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:43.925191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:44.160520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162df8c-5a98-4b5f-9fc2-1242d30ddc46 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:44.185049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb9117-2e72-4b6e-9867-9d317dcad30c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:44.207159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:44.208393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.209716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.209716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.210311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:44.210747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:44.215435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd3223-02f8-480d-bc39-c24e0c5e52c1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:44.221308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 180/377] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:43 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:44.243881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2023318-90a4-4cf6-bfaf-7b3340cabc9e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.244317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2023318-90a4-4cf6-bfaf-7b3340cabc9e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.244697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2023318-90a4-4cf6-bfaf-7b3340cabc9e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.246704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 181/378] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:44.255985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:44.256549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.256934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.257174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.257553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.258136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.258809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] PATH_INFO: `/v3/projects/ff9660c7d5b94c33b26aef6fd109810a` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.260258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] RBAC: Authorizing `identity:get_project(project_id=ff9660c7d5b94c33b26aef6fd109810a)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:44.264449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:44.267182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 198/379] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:43 2022] GET /identity/v3/projects/ff9660c7d5b94c33b26aef6fd109810a => generated 344 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:44.276360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:44.284120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:44.285721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:44.302006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:44.374468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 182/380] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:44.378581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:44.441026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ff747-a202-433f-b5a4-d4deab18791b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:44.489044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:44.489458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.490853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.492409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] PATH_INFO: `/v3/groups/ec46316d59524cc5ae60a077a6519d2a` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.503006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] RBAC: Authorizing `identity:get_group(group_id=ec46316d59524cc5ae60a077a6519d2a)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:44.504510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fcf40-d517-4eaa-9d27-fa81884c9da2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:44.507710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 199/381] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:44 2022] GET /identity/v3/groups/ec46316d59524cc5ae60a077a6519d2a => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:44.518041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:44.672606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704e3a-b728-4682-ae22-d521118917a1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:44.717818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:44.718370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.718649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.718919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.719676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] RBAC: Adding query filter params (group.id=ec46316d59524cc5ae60a077a6519d2a, role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ff9660c7d5b94c33b26aef6fd109810a) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:44.719983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:44.721637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb403a-8c71-41a2-9d0e-cf1248bdc816 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:44.732202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 183/382] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:44 2022] GET /identity/v3/role_assignments?group.id=ec46316d59524cc5ae60a077a6519d2a&scope.project.id=ff9660c7d5b94c33b26aef6fd109810a&role.id=edae49896b104fddabf7a02644f5d914 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:44.752616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019326d4-d345-4662-b3f5-8a1b9f651306 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.752946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019326d4-d345-4662-b3f5-8a1b9f651306 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.753231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019326d4-d345-4662-b3f5-8a1b9f651306 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.754967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 200/383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:44.761153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b903c-3283-4eec-bdc1-8e337009192b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:44.761465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b903c-3283-4eec-bdc1-8e337009192b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:44.761741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b903c-3283-4eec-bdc1-8e337009192b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:44.787457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cc6b903c-3283-4eec-bdc1-8e337009192b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:44.788828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cc6b903c-3283-4eec-bdc1-8e337009192b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:44.792420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:44.837619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019326d4-d345-4662-b3f5-8a1b9f651306 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:44.988927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 184/384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:45.010534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019326d4-d345-4662-b3f5-8a1b9f651306 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:45.067369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fb97-9ace-4d69-bcdf-aca9e671dc6c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.067722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fb97-9ace-4d69-bcdf-aca9e671dc6c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.068487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fb97-9ace-4d69-bcdf-aca9e671dc6c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.069696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:45.070291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 185/385] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:45.077204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.077875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.078358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.081394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:45.082597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.082988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.083465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.102291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:45.103632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:45.105343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:45.107002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 201/386] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:45.110973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.120867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:45.122127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:45.321942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 186/387] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:45.323636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.390146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8f55-034f-4081-a679-8ed7a233c24c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:45.482783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:45.487707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:45.488341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.493902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.494250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.494990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:45.495299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:45.496718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:45.520321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 202/388] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:45.527589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.555079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1eec3-9961-48fb-a1f4-9ec8cca90fc4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:45.598009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:45.598659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.598933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.602004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.632998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:45.635652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:45.643594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:45.645860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:45.648102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 187/389] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:45.653789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.698532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5faa-7d2c-44da-af73-8d291489b4de None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:45.760793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:45.766625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.767108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.767584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.770283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:45.772423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:45.774499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 203/390] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:45.775404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.899567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be930898-0a60-427b-80c2-f77c7c369a16 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:45.951192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:45.958380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:45.959881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:45.960366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:45.960837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:45.961888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:45.962404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:45.964110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:45.974336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 188/391] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:45.975615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:45.987042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379822a6-f264-4a2d-a155-4090e8bd9f00 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.020757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:46.064154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.065747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.066395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.066701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.080837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:46.089056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:46.090157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:46.091639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 204/392] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:46.092761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.160567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b24fc-9563-42f5-a34c-835abcf0b811 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.217669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.218842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.218842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.219269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.220678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.222343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.226521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d4d65-0555-4e01-bb99-acada02ed59c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.230179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 189/393] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:45 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:46.230922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.270243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.270886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.271440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.271856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.273225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.274897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.277270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 205/394] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:46 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:46.278516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.314482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:46.366168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:46.368744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861e28f-372f-4fc1-a389-16b2bb7d04f0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.414376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.415442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.415674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.416083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.417008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:46.417445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.419184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.429963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 190/395] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:46.431077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.457579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8b0e8-7abf-4e3e-af04-808696e09935 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.517489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.518873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.519604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.520326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/groups/b7e6b12af5d445c29e304e610773e89c/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.523581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, group_id=b7e6b12af5d445c29e304e610773e89c, role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.525961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.549427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 206/396] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 6 11:35:46 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/groups/b7e6b12af5d445c29e304e610773e89c/roles/acebbc3107e1448599b7b2bd0096b755 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:46.551469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.635951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7401a-7919-43cd-a175-aefa9118b218 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.703240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:46.752993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.753675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.753981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.754254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.755209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4dcf-54d5-490a-ac43-78365ea9c017 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.755724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.757459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f868-7a0d-431e-9ffb-bc11f3f8b65f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.759562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 191/397] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:46 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:46.760867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9eb5-ea0d-4bd5-a759-e09186d9dbf3 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.761154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9eb5-ea0d-4bd5-a759-e09186d9dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.761421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9eb5-ea0d-4bd5-a759-e09186d9dbf3 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.767469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 192/398] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:46.768172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9eb5-ea0d-4bd5-a759-e09186d9dbf3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.773608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:46.819386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.821190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.821904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.822424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.825931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.828909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03cb5-b550-4fdf-928b-236ac7ada90e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.833222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 207/399] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:46 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:46.836854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.837362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.838404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.868957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:46.870370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:46.912162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9eb5-ea0d-4bd5-a759-e09186d9dbf3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:46.951993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 208/400] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:46.954819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:46.955508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:46.955788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:46.956062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/groups/b7e6b12af5d445c29e304e610773e89c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:46.958185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f, group_id=b7e6b12af5d445c29e304e610773e89c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:46.959084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:46.959753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:46.980490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 193/401] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 6 11:35:46 2022] PUT /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f/groups/b7e6b12af5d445c29e304e610773e89c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:47.097195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd85c9-daa0-49c5-b765-e705785648ac None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:47.107210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:47.141159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:47.141860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.142141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.142417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.143802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:47.145417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:47.147830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 209/402] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:46 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:47.153859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:47.217537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:47.285380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618153-28b5-4a6f-b421-c50abc037da2 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:47.330003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:47.330701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.330982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.331262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/groups/b7e6b12af5d445c29e304e610773e89c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.333393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, group_id=b7e6b12af5d445c29e304e610773e89c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:47.335547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:47.353335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:47.356369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 194/403] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 6 11:35:47 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/groups/b7e6b12af5d445c29e304e610773e89c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:47.526625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af859084-73db-4ad8-b9e2-a1c29096c876 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:47.592867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:47.593822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.594102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.594428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.610648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9eff1025-aedf-46dc-a8a9-a79c3914c4ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:47.617741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:47.618688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:47.619546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:47.624597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 210/404] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:47.626626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:47.674230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f174eda-6831-4b1b-8a1d-ff7f59130dfc None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.674562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f174eda-6831-4b1b-8a1d-ff7f59130dfc None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.674785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f174eda-6831-4b1b-8a1d-ff7f59130dfc None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.676745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 211/405] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:47.682640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.682926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.683181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.706072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:47.707791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:47.758343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:47.816431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 212/406] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:47.904872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e487a9-9212-4e4a-a803-f631128aec95 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:47.922883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:47.960970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:47.962075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:47.962551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:47.963013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:47.963893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:47.964413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:47.966065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:47.977498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 195/407] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:47.985693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:48.157119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b5ed-bdcf-4c02-8540-595f867a8f9e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:48.212108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:48.212772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:48.213055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:48.213325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:48.214542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:48.216011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:48.221785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 213/408] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:47 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:48.225289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:48.284696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:48.358660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:48.460122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a302475-05c9-44f6-924d-08aac5cde0e0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:48.526732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30228fd-ac00-4302-9d1d-29eb7cb57114 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:48.563466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:48.564127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:48.564407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:48.564693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:48.570582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:48.572130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:48.574316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 196/409] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:48 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:48.579945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:48.616676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:48.618685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:48.619458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:48.620206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:48.650910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:48.656056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:48.657801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:48.659445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:48.666118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 214/410] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:48.672564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:48.695068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:48.831156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c031e5-8a9a-4798-a206-99169383eaac None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:48.884373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:48.900974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921689-7718-413d-85c8-4d4cb44ae81f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:48.916045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:48.916738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:48.917006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:48.917284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/groups/b7e6b12af5d445c29e304e610773e89c/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:48.923699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed1c92f192bb48ac8bdf6281b2882c13, group_id=b7e6b12af5d445c29e304e610773e89c, role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:48.925194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:48.949326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 197/411] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 6 11:35:48 2022] PUT /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13/groups/b7e6b12af5d445c29e304e610773e89c/roles/acebbc3107e1448599b7b2bd0096b755 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:35:48.990580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:48.991212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:48.991486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:48.991754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:48.992491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:48.992795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:48.994307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:49.004520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 215/412] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:49.014273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:49.173541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be88ec6-fea0-4347-9a33-0d3ea7450ae1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:49.182719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:49.217123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:49.217826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.218099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.218373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] PATH_INFO: `/v3/groups/ec46316d59524cc5ae60a077a6519d2a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.219588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] RBAC: Authorizing `identity:get_group(group_id=ec46316d59524cc5ae60a077a6519d2a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:49.221082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c17ac3-8afd-40fc-a410-eedefb0bb496 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:49.223167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 198/413] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:49 2022] GET /identity/v3/groups/ec46316d59524cc5ae60a077a6519d2a => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:49.233042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:49.289471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09604028-51aa-45cd-8696-759cbb785b5d None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.289839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09604028-51aa-45cd-8696-759cbb785b5d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.290140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09604028-51aa-45cd-8696-759cbb785b5d None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.292353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 199/414] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:49.308855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf27dc-4bb9-405b-8a0c-64b9f9f57cfa None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.309777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf27dc-4bb9-405b-8a0c-64b9f9f57cfa None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.311571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf27dc-4bb9-405b-8a0c-64b9f9f57cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.352922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a2cf27dc-4bb9-405b-8a0c-64b9f9f57cfa None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:49.355535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a2cf27dc-4bb9-405b-8a0c-64b9f9f57cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:49.489129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b0ca3-0712-4e6e-8aa4-863fd6a3ece6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:49.572176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:49.572890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.573147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.573373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] PATH_INFO: `/v3/projects/ff9660c7d5b94c33b26aef6fd109810a` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.574828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] RBAC: Authorizing `identity:get_project(project_id=ff9660c7d5b94c33b26aef6fd109810a)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:49.577929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:49.579730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 216/415] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:49 2022] GET /identity/v3/projects/ff9660c7d5b94c33b26aef6fd109810a => generated 344 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:49.582375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 200/416] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:49.587282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:49.640011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:49.750384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585171-61a4-4fba-ab86-0f27801d7312 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.751134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585171-61a4-4fba-ab86-0f27801d7312 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.751635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585171-61a4-4fba-ab86-0f27801d7312 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.755818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 201/417] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:49.765215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.765524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.765810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.795723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fccc31-58a9-4435-85d0-4ea1e7b3b532 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:49.811110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:49.812520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:49.874614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:49.875628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:49.875906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:49.876170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] PATH_INFO: `/v3/projects/ff9660c7d5b94c33b26aef6fd109810a/groups/ec46316d59524cc5ae60a077a6519d2a/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:49.881581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff9660c7d5b94c33b26aef6fd109810a, group_id=ec46316d59524cc5ae60a077a6519d2a, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:49.884678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:49.921701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 202/418] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:49.924369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 217/419] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 6 11:35:49 2022] PUT /identity/v3/projects/ff9660c7d5b94c33b26aef6fd109810a/groups/ec46316d59524cc5ae60a077a6519d2a/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:35:50.010482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:50.075085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:50.207163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc17b-bc96-45b8-ad72-3a3fa81d7bb3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:50.289953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:50.290497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:50.290772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:50.291038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:50.311847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:50.317351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:50.319224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:50.320195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 203/420] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:50.330973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:50.587428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:50.602273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:50.644871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99277891-c8d9-46bf-8fa4-bee4c3ee9c46 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:50.728647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:50.730168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:50.730938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:50.731724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:50.733261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:50.734132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:50.741293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e74584-0e03-4955-9040-5d1cc28d0875 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:50.757866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 218/421] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:50.760468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8be1a-5eb9-4c71-9c11-f3ff93e114a9 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:50.761286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8be1a-5eb9-4c71-9c11-f3ff93e114a9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:50.761992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8be1a-5eb9-4c71-9c11-f3ff93e114a9 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:50.764752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 219/422] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:50.771099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8be1a-5eb9-4c71-9c11-f3ff93e114a9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:50.816028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:50.842927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6f82-2c68-427d-ac22-6577b85abff9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:50.909690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:50.910884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:50.910884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:50.911151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:50.931222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-fd6ca95a-c2d4-4baf-ba75-00be7305f9dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:50.941505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:50.943865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:50.945626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 204/423] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:50.946911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:50.947860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:50.947860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:50.980677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:50.982704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:50.983640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8be1a-5eb9-4c71-9c11-f3ff93e114a9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:51.046541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:51.047968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.048659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.049427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.077124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:51.118257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 205/424] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:51.119757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:51.200233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:51.206548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b32f0-642c-47ba-ae98-1d2b6e1ad9bb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:51.340178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:51.373468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c9b6e-f859-456f-8fc9-e6355cfd32c1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:51.389191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 220/425] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:51.391643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158551cf-d7bc-4ce1-8f5f-24d79a9d3856 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.392409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158551cf-d7bc-4ce1-8f5f-24d79a9d3856 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.392873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158551cf-d7bc-4ce1-8f5f-24d79a9d3856 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.397748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 221/426] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:51.398962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158551cf-d7bc-4ce1-8f5f-24d79a9d3856 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:51.420917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:51.421899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.422369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.423030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.423954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] RBAC: Adding query filter params (name=member) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:51.424490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:51.426200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c641ac-4717-46de-8c55-a0e8f7e8ef48 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:51.441771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 206/427] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:51.441771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.441771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.441771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.466081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:51.467386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:51.474981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:51.561444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158551cf-d7bc-4ce1-8f5f-24d79a9d3856 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:51.573668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 207/428] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:51.573668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:51.640442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:51.642133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.642420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.642694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.645261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:51.646999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:51.649159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 222/429] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:51 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:51.650045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:51.686028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:51.752560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8260-d491-4c79-9d4b-c100f41016d3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:51.804399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:51.805195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.805576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.805980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.826019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5916a7-ee78-430d-98af-b749fef12437 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:51.844314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:51.873681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:51.875325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:51.875599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:51.875868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:51.890855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:51.896772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:51.898789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 223/430] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:51.900477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:51.949322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:51.951741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.062203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b01e6e-96f4-4366-843f-56edea3401bd None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.071678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.130015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.131691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.132132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.132566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.133314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 208/431] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:52.134926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.137738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.138857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.141094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 224/432] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:51 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:52.142356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.226813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.262643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83804316-9992-456b-9c51-4b2dd607ce81 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.304244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.304898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.305174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.305451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.320456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:52.326019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:52.326978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:35:52.327769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:52.331337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 209/433] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:52.331337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.349054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37820adb-eb69-40ab-8802-5ac843cade00 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.371629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.401298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.402236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.402667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.403098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.403920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:52.404372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.405972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.421230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 225/434] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:52.422478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.470916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255646f-0551-4c51-a3ad-889315e2d636 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.514770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.515421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.515697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.515965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] PATH_INFO: `/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.517180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] RBAC: Authorizing `identity:get_project(project_id=d7e2d0ef44f640f9a602ff7c1ab0e74f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.518725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.520481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 210/435] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/projects/d7e2d0ef44f640f9a602ff7c1ab0e74f => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:52.521374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.561683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1c93-f36b-46c3-8176-229ef25fcfd6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.563803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.613566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.614650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.615659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.616131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.617171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:52.617508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.619024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.625457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 226/436] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:52.626482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.664108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.672589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe25af-ef6c-40e1-a4c8-10fb89425600 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.748115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.750122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.750490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.750931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.752570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:52.753073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.755893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.774422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 211/437] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:52.776028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:52.801705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24f1e-d253-4bea-9687-9e81c57187d7 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.871369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.872015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.872291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.872563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.913507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:52.945265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ee0c-5efb-46aa-9a64-c5ce2871053f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:52.970523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:52.972035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:52.994114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:52.995209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:52.995698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:52.996181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:52.998275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:53.001082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748065-d27c-49e4-9e3a-bdfbc7881ed8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:53.010201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 212/438] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:53.010201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4112-ac63-4ea9-a3c8-5337e1237764 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.010201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4112-ac63-4ea9-a3c8-5337e1237764 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.010201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4112-ac63-4ea9-a3c8-5337e1237764 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.011612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 213/439] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:53.013099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4112-ac63-4ea9-a3c8-5337e1237764 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.062751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:53.115251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 227/440] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:53.116755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.180988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4112-ac63-4ea9-a3c8-5337e1237764 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.200374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:53.234953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:53.235976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.236462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.236949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.241947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19911e-e545-4973-b350-4f906d73fff2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.294607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:53.295338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.295682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.296072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.296968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:53.297443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:53.299153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053e3d1-a1f8-438a-b8a1-1d9458d06ef8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:53.306502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 228/441] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=d7e2d0ef44f640f9a602ff7c1ab0e74f&role.id=0da2f51a570245b1acf832a45de984cc => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:53.308664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.309115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.309547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.341959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:53.344246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:53.348373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:53.349996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:53.391206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:53.487653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 229/442] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:53.487653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.496979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 214/443] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:53.503118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.621597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:53.693292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34d6ac-ed2a-4ddc-9b57-8c087be12ae7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.701754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892e0c4-6eb6-4405-a49d-c68d118b10d7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.757663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:53.767989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:53.769587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.770488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.771429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.773816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:53.775117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:53.775797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:53.776109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:53.776410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] PATH_INFO: `/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:53.776753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:53.778118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] RBAC: Authorizing `identity:get_project(project_id=ed1c92f192bb48ac8bdf6281b2882c13)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:53.779610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:53.780288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 230/444] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:53.781670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 215/445] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/projects/ed1c92f192bb48ac8bdf6281b2882c13 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:53.781851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.789960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:53.953478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e7db2-9abe-4c10-a395-040186b429e5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.956643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b5dd-aa5e-4d57-b64b-991cbcdded89 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:53.989468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:54.001396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.002922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.002922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.002922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.022071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.023174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.023627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.024102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.024725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:35:54.026228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.028745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.032350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 216/446] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:54.033956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:35:54.035134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:54.036156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:35:54.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 231/447] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:54.043388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:54.111356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:54.168470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ddac9-fbcf-4a7c-bfff-ab6ed6b8defe None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:54.175823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19359b4e-f823-4b68-9e5b-7978026c43a2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:54.216403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.217020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.217300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.217569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.218344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=0da2f51a570245b1acf832a45de984cc, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:54.218650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.220154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.227560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 232/448] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=0da2f51a570245b1acf832a45de984cc => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:54.228439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:54.232285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.232949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.233224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.233492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] PATH_INFO: `/v3/groups/b7e6b12af5d445c29e304e610773e89c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.234853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] RBAC: Authorizing `identity:get_group(group_id=b7e6b12af5d445c29e304e610773e89c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.236406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.238232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 217/449] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/groups/b7e6b12af5d445c29e304e610773e89c => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:54.248932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:54.317717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:54.389198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d7dd-51d2-4042-98aa-8f15a2b59c86 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:54.442784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.443479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.443756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.443912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7d4e-e586-4b3e-b293-aa2e82072366 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:54.444083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.444712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:54.445014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.446505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.456380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 233/450] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:54.463405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:54.504977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:54.516335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.517722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.517915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.518254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.519126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] RBAC: Adding query filter params (group.id=b7e6b12af5d445c29e304e610773e89c, role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:54.519748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.522759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.533180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 218/451] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/role_assignments?group.id=b7e6b12af5d445c29e304e610773e89c&scope.project.id=ed1c92f192bb48ac8bdf6281b2882c13&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 611 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:54.650715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882d6b-88ca-43b6-a784-143e7a1a1b8d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:54.671487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:54.705123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:54.706235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:54.706662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:54.707163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:54.795062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:54.796575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:54.936970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 234/452] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:35:54.944974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:55.078003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1002b-73ae-4dac-9aa1-01a8f6c092c6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:55.119184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:55.119837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:55.120113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:55.120386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] PATH_INFO: `/v3/projects/ff9660c7d5b94c33b26aef6fd109810a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:55.121633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] RBAC: Authorizing `identity:get_project(project_id=ff9660c7d5b94c33b26aef6fd109810a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:55.123163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:55.124929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 219/453] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:35:54 2022] GET /identity/v3/projects/ff9660c7d5b94c33b26aef6fd109810a => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:55.131504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:55.149059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:55.243332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5cb4-8b7f-4d37-a6f8-ef55012674ba None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:55.283689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:55.284337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:55.284617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:55.284890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] PATH_INFO: `/v3/groups/ec46316d59524cc5ae60a077a6519d2a` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:55.286120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] RBAC: Authorizing `identity:get_group(group_id=ec46316d59524cc5ae60a077a6519d2a)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:55.288354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b95969-c832-4847-9e01-0d82f664f70b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:55.290023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 235/454] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 6 11:35:55 2022] GET /identity/v3/groups/ec46316d59524cc5ae60a077a6519d2a => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:55.295977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:55.362591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:55.413128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d33f-b93f-4a45-9460-fd9c089e4a00 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:55.450450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:55.451070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:55.451344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:55.451613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:55.452351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] RBAC: Adding query filter params (group.id=ec46316d59524cc5ae60a077a6519d2a, role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ff9660c7d5b94c33b26aef6fd109810a) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:55.452655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:55.454156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6291516-c32a-457c-b5a4-58804f4dcf4a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:55.461049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 220/455] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 6 11:35:55 2022] GET /identity/v3/role_assignments?group.id=ec46316d59524cc5ae60a077a6519d2a&scope.project.id=ff9660c7d5b94c33b26aef6fd109810a&role.id=edae49896b104fddabf7a02644f5d914 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:35:58.195090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9657945-3d4d-4eb9-9ec4-0c8b0389e431 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.195653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9657945-3d4d-4eb9-9ec4-0c8b0389e431 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.195925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9657945-3d4d-4eb9-9ec4-0c8b0389e431 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.198569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 236/456] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:58.204525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5023a-96d5-4c5c-8c08-540f155c80a8 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.205076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5023a-96d5-4c5c-8c08-540f155c80a8 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.205076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5023a-96d5-4c5c-8c08-540f155c80a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.239504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e0b5023a-96d5-4c5c-8c08-540f155c80a8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:58.242141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e0b5023a-96d5-4c5c-8c08-540f155c80a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:58.363011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 221/457] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:58.579178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb2d12-f6ec-44ec-a13e-4a0ee3b1c49b None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.579354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb2d12-f6ec-44ec-a13e-4a0ee3b1c49b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.579639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb2d12-f6ec-44ec-a13e-4a0ee3b1c49b None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.581326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 237/458] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:35:58.587458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.588053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.588499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.592718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e925fe5-1408-4c5e-98a9-8d4805544d58 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.593338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e925fe5-1408-4c5e-98a9-8d4805544d58 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.593837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e925fe5-1408-4c5e-98a9-8d4805544d58 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.596037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 238/459] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:35:58.616412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:58.616412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:58.616412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:58.624542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:58.626343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:58.651585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:35:58.653766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:35:58.791870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 222/460] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:35:58.792822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:58.871866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 239/461] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:35:58.941615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:59.102558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872d4cf-b061-4c93-becc-c9fc12c8c3d9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:59.160774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:59.162506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:59.163271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:59.163862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:59.178566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:59.180518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:35:59.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:59.210297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:35:59.211213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:35:59.211213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:35:59.211213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:59.217787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 223/462] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:35:59.221089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:59.271082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:59.349463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:59.419972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:59.465120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd987-58a4-426a-888d-d9c37cbbede0 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:59.515288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:59.516868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:59.517614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:59.518155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:59.519044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:35:59.519566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:59.522002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:59.545752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 240/463] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:35:59.549437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:59.674085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414dbd6-a87a-4890-8505-94cb3b0216cf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:59.687772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:35:59.746204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74daa7-5cf0-4f68-98ac-9727ef07969d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:35:59.752307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:59.753807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:59.754509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:59.755193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:59.780614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:59.784016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:59.808470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:35:59.809217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:35:59.809497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:35:59.809796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:35:59.810495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:35:59.812640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:35:59.816652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:35:59.819819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:35:59.822373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:35:59.822373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:35:59.822373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:35:59.825935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 224/464] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:35:59.828423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:35:59.845805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 241/465] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:35:59 2022] POST /identity/v3/users => generated 248 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:35:59.846753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:00.009668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986281-a838-42f7-8184-09f6d9d144a6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:00.044453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:00.055391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:00.056825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.057264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.057661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.058659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:00.058820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:00.062016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb278c7-adc8-41f5-b98f-543a726df8e5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:00.084980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 242/466] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:00.087500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fddc-d409-4510-b2f7-d89c1fcfc365 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.088073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fddc-d409-4510-b2f7-d89c1fcfc365 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.088673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fddc-d409-4510-b2f7-d89c1fcfc365 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.091770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 243/467] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:00.093258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3fddc-d409-4510-b2f7-d89c1fcfc365 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:00.119805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:00.133196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590dfe-09e6-418b-9b6e-fa0600d4f144 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:00.214975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:00.217150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.217967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.218682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.235158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:00.236676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-e0850356-1263-40e6-a848-4e9979a5898a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:00.267212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:00.269278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:00.270360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:00.270360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:00.270360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:00.270360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 225/468] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:00.272902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a708ce-0604-45fc-b95b-d660ad0cd404 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.273378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a708ce-0604-45fc-b95b-d660ad0cd404 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.273856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a708ce-0604-45fc-b95b-d660ad0cd404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.295672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3fddc-d409-4510-b2f7-d89c1fcfc365 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:00.309396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-45a708ce-0604-45fc-b95b-d660ad0cd404 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:00.311402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-45a708ce-0604-45fc-b95b-d660ad0cd404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:00.377710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:00.379165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.379971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.381314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.382971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:00.386637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:00.432101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 244/469] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:36:00 2022] POST /identity/v3/users => generated 248 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:00.434984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:00.462894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 226/470] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:00.549548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:00.558095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1279-c156-45f1-94c1-018f93f3b8e7 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.558690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1279-c156-45f1-94c1-018f93f3b8e7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.559240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1279-c156-45f1-94c1-018f93f3b8e7 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.562716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 227/471] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:00.569566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.570194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.570786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.626896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:00.628442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:00.666084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a403f6-970b-4a82-9476-6d100d00d00b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:00.716900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 228/472] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:36:00.717999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:00.718842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.719297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.719738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.720616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:00.721098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:00.722710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:00.737936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 245/473] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:00.749125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:00.758141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:00.846166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:00.889375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a64680-36ce-4764-b8f8-8cc6600bbd38 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:00.960389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:00.961457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:00.962032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:00.962515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:00.964735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:00.967814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:01.014915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff46b4-8b55-4e2b-8ae5-6c1e994549ed None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:01.018900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:01.035685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 229/474] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:36:00 2022] POST /identity/v3/users => generated 246 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:01.067116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:01.068407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:01.068917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:01.069374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:01.085008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:01.087363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:01.108960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:01.110114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:01.111116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:01.111116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:01.111116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.113281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 246/475] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:01.124124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:01.153450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:01.294212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:01.351580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137281b4-0e1f-4b9c-a9c2-eb48472b3fa7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:01.370226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f4b6-7b16-4f3f-b833-78072fbb7bcd None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:01.413507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:01.414634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:01.415111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:01.416661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:01.418483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:01.418967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:01.422533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:01.436367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:01.436998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:01.437276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:01.437548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:01.438293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:01.439732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:01.453694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.sql.core [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 11:36:01.453694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 11:36:01.453694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [parameters: {'id': '8421a788bf5042c8ad7b14d97301c4fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 11:36:01.453694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87997) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 11:36:01.457711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 230/476] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:01.468372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application result.read() Sep 06 11:36:01.471331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:36:01.473273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application yield resource Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise self.value Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application session.commit() Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 11:36:01.476144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.session.flush() Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application compat.raise_( Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 11:36:01.478064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:36:01.479792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application result.read() Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:36:01.482485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [parameters: {'id': '8421a788bf5042c8ad7b14d97301c4fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:01.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 11:36:01.487059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.489584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 247/477] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 6 11:36:01 2022] POST /identity/v3/roles => generated 144 bytes in 320 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:01.489584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:01.549931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:01.693363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d02d3-8d04-45be-b8e7-256169f900e3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:01.717003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70ce2c-2ff2-4508-afeb-efb460f65bfe None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:01.749055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:01.807947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:01.809002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:01.809498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:01.810020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:01.813712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:01.816468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e21a-7e84-4c22-ac57-abc9a5db972f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:01.817873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:01.819027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:01.819512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:01.819998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:01.863784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:01.885255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:01.886533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 11:36:01.887348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:01.890980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 248/478] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:36:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:01.895119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 231/479] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:36:01 2022] POST /identity/v3/users => generated 249 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:01.904890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:02.126358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:02.150908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd49d-dcd6-4513-9909-2675ed930ec8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:02.201132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:02.201840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:02.202122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:02.202391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:02.203083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:02.203385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:02.204804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c06d4-bad8-4ae4-a9a6-796a892ca41e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:02.216166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 249/480] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:02.915753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98f84-2b75-46c4-9590-3febee0b15ae None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:02.916815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98f84-2b75-46c4-9590-3febee0b15ae None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:02.917329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98f84-2b75-46c4-9590-3febee0b15ae None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:02.919283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 232/481] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:02.925899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7364-3ee0-43db-b9ef-4f8092408e1b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:02.926415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7364-3ee0-43db-b9ef-4f8092408e1b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:02.926843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7364-3ee0-43db-b9ef-4f8092408e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:02.975992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4f0d7364-3ee0-43db-b9ef-4f8092408e1b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:02.978131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4f0d7364-3ee0-43db-b9ef-4f8092408e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:03.095500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 250/482] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:36:03.733700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf98f84-2b75-46c4-9590-3febee0b15ae None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:03.931864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad224-47be-4ce6-80ef-e806cb93e75e None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:03.932395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad224-47be-4ce6-80ef-e806cb93e75e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:03.932821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad224-47be-4ce6-80ef-e806cb93e75e None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:03.933078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf98f84-2b75-46c4-9590-3febee0b15ae None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:03.941881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 251/483] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:03.945292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:03.945914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:03.946464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:03.986522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:03.990405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:04.023891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:04.024914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:04.026289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:04.026784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:04.070402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:04.080291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:04.081490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:04.082389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 233/484] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:04.092403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:04.199884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 252/485] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:36:04.218082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:04.398197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cc76-a811-4b1f-be8e-4f869eab60b2 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:04.495763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:04.496857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:04.497324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:04.497823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:04.499104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:04.499620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:04.502381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb28d30-3af1-4de6-a743-9912a05b3c6d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:04.524913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 234/486] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:04.533374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:04.539039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:04.815755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977866b-9385-4f5b-99d6-42f5819a39b0 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:04.909263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:04.910373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:04.910850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:04.911321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:04.940552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:04.943948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:04.946360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f678a0-d034-4082-937b-19b5be856b71 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:04.946869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f678a0-d034-4082-937b-19b5be856b71 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:04.947271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f678a0-d034-4082-937b-19b5be856b71 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:04.950239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 235/487] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:04.960749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:04.961190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:04.961628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:04.973881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:04.975496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:04.976347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:04.976347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:04.976347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:04.979484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 253/488] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:04.980748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.003891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:05.006668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:05.088381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.141019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 236/489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:05.142460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.167421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bbd48-0381-4f5b-a938-6c7d941a361c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:05.210061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.229707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:05.230410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.230685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.230953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.244080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-408bc7cc-bd39-4b75-ae26-e2bdccc1c024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:05.249813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:05.252131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 254/490] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:05.253676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ad1a6-64c8-421f-8555-b8c916be6973 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.256571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ad1a6-64c8-421f-8555-b8c916be6973 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.256571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ad1a6-64c8-421f-8555-b8c916be6973 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.256571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 255/491] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:05.261902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ad1a6-64c8-421f-8555-b8c916be6973 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.364178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd8177-1c47-44a6-a68c-644a7ca83780 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:05.398239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ad1a6-64c8-421f-8555-b8c916be6973 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:05.417423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.430183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:05.431180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.431457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.431899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.433074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:05.433581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:05.436219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f483c-4cc8-446a-844a-feb2ebabe261 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:05.445455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:05.446127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.446407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.446691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.447381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:05.447690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:05.449944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf110c-1800-4f69-a936-040c33f2c941 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:05.463536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 237/492] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:05.463536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.463536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.463536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.470409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 256/493] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:05.478343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7c13-dab9-4900-bbd8-fec132d84d7c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.478614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7c13-dab9-4900-bbd8-fec132d84d7c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.478886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7c13-dab9-4900-bbd8-fec132d84d7c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.481078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 257/494] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:05.483410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb7c13-dab9-4900-bbd8-fec132d84d7c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.496982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:05.498484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:05.603114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 238/495] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:05.604312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.625798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.675521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb7c13-dab9-4900-bbd8-fec132d84d7c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:05.732345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:05.733003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.733276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f1adf-7a44-44d5-83a0-73734673d603 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:05.733472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.733605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.743524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:05.745059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-3e8a3818-eb00-4894-b293-27ef809289cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:05.761412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:05.762452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:05.763609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:05.767623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 258/496] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:05.768957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.769242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.769522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.784774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:05.785427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:05.785735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:05.786010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:05.790258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:05.791516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:05.801040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:05.803308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:05.823711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:05.825494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:05.828096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:05.828096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:05.828096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:05.828096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 239/497] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:05.828096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.833087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.889990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 259/498] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:05.890800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:05.925036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:05.979726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2b77-591b-4d77-ac41-4161f247e018 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.023555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.024336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.024599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.024867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.036621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:06.042085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:06.043226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 240/499] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:06.044230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.062970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ad6ba-7854-47f8-99ce-74178f407a66 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.121323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.122321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.122765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.123219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.124500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:06.124929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.127447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.187367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 260/500] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:06.188386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.197114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7345a7-b6ed-44b3-8d2c-5ddd33debdd4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.219236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.230249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.237762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.238709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.239072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.239414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.240907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:06.241443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.243025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.265265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 241/501] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:06.266410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.309772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900de5-1a43-4982-a7d7-f27939f5b43a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.347678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.348571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.349012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.350205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.365960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:06.373602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:06.374761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:06.375790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 261/502] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:06.376569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.428076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac9cbd-94fb-4307-a428-9a8f5d592f53 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.465111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.465733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.465949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.466221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.466906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:06.467209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.468648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.478160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 242/503] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:06.478160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.489279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.508429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f649dc-1c73-4599-a18e-abf3549cd04f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.540122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.546949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.547569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.547855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.548128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.598241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9dd6-4fe0-4c93-8704-89bbab3c2717 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.636932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.637568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.637876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.638154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.644112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.645744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.650727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.652238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:06.668917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:06.669951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:06.670815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:06.670815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:06.670815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:06.670815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 243/504] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:06.671033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.761467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.791577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 262/505] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:06.793031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.806858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915e94-9e8d-40ce-b1de-f86fcfb184dc None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.809686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:06.846368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.847006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.847283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.847554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.857308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.858835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:06.873584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:06.875028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 244/506] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:06.876591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:06.932998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545b061-36cf-4616-a8b3-f0ab50ba13b3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:06.972705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:06.973336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:06.973614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:06.973922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:06.975059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:06.975369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:06.976817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:06.986719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 263/507] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:06.987605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.026632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ecf62-7f76-4eac-844e-2b1c10cae035 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.066430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.067210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.067633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.068060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.072324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.085225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.087577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:07.109350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:07.111351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:07.112918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:07.112918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.112918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:07.112918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 245/508] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:07.112918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.144719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.158200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11790b-1aff-4659-b571-df3ff5682136 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.204715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.205540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.205946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.206305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.207186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:07.207656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.209311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.226344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 264/509] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:07.227794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.274349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e07961-472e-4596-924d-c2f4f05e739d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.320882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.321832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.322261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.322735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.340427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.342982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:07.361440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:07.363380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:07.365392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:07.365392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.365392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:07.365392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 246/510] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:07.365392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.387331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d249b-82cd-4775-88e6-eeef1548e2e8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.403486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.439409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.455229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.456285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.456778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.457257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.458778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:07.460143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.462990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.479580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6240af-01a0-472f-888c-bef5c30f92c1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.498249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 265/511] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:07.499517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.524166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.524817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.525090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.525359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.539443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.541098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:07.568661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:07.570357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:07.578230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:07.578230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:07.578230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:07.578230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 247/512] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:07.578230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.634114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295abebd-cdd5-4848-9324-f0ebc872fefd None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.673411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.674076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.674356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.674637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.675306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:07.675611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.677070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.690584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 266/513] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:07.691611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.726328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.741397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367bcc8-5f13-4881-b7a2-bae77de36511 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.775686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:07.797631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.798288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.798567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.798840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.799506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.800986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.826527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8bb32-41bb-4e37-9567-f9be48283b78 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:07.839670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 248/514] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:36:07 2022] POST /identity/v3/users => generated 247 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:07.839670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:07.892927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:07.893929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:07.894360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:07.894784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:07.895855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:07.896948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:07.899346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:07.924393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 267/515] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:07.925832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.009436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695fd03-0677-4851-b0a9-e2c3836ffbd4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.076907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.078902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.079537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.079855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.088885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.123702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ae1e-af7e-4eed-9b7e-cb8eca7649ad None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.158820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.183626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.184679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.185181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.185847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.186950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:08.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.190248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.190473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.191792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.215292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 268/516] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:08.216685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.350217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 249/517] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:08.351677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.423750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847147b-b6e4-447f-8b4a-14b535e9bd3d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.468969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.469995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.470498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.470945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.482631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.484399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:08.500716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:08.502590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 269/518] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:08.512876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.517141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.520334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.572677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421575-295d-4d84-a7b7-1ed68ccddfef None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.640074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.641144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.641628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.642148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.656615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.659428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:08.684295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:08.686087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 250/519] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:08.688087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.707102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7d743-971d-47b3-8b63-02333f326948 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.769252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.770597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.771092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.771569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.786485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.789877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:08.803659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:08.805003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 270/520] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:08.806524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.850778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c6ad-0b46-42f3-932f-f44409ff0d4a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:08.891801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:08.892445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:08.892705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:08.892977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:08.896092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.904719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:08.907944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:08.912834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:08.934456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:08.935995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:08.938456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:08.938456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:08.938456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:08.938456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 251/521] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:08.939935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:08.973789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c8b5-161d-4b67-9144-3d4480e0974e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.035698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.036384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.037529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.037529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.037997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:09.038289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.039765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.068557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 271/522] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:09.068557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.097056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a04ee3-d876-42d2-82d1-0b26d005ad5f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.149370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.150146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.150429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.150694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.151418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:09.151716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.153218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.178796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 252/523] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:09.178796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.225782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667abcb-0ee5-4d97-84ad-128389713c75 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.244095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.275469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.294382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.295446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.295970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.296491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.297955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:09.298626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.300081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.323994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 272/524] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:09.324848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.347028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c83ab-3df9-4219-835e-59a31e9e0f4d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.405355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.406811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.407254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.407822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.409016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:09.409618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.412147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.439940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 253/525] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:09.439940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.487107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0609848-6eb7-42a1-a6d4-d6ad72939a4c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.548284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.549169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.549628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.550161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.597259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8db7b8-68e4-4938-b2a4-692db0a76dae None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.609084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.623490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.647925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.648837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.649247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.649694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.650913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=7739336ec72248feb11d1d6a7a8de166) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:09.651365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.654038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.657348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.659452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:09.663717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 254/526] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/role_assignments?user.id=7739336ec72248feb11d1d6a7a8de166&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:09.664926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.812325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 273/527] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:09.813923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:09.820602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e2a-9ad6-4e42-bfd9-c35be3ef8aa7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.865565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:09.866614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:09.866933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:09.867167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:09.932600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.951942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:09.981937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd1bd-8be3-4b4b-a0e3-ff7449403d8b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:09.995017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:09.997714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3ec2b-b45e-4d49-b15a-06bb5ff4ee63 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.029908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:10.030750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.031134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.031510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.041850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:10.043487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:10.058933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:10.059916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:10.060760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 274/528] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:10.061231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:10.166063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 255/529] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:10.168630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b264-1a65-4e94-ab3b-c617396ccfe9 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.169097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b264-1a65-4e94-ab3b-c617396ccfe9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.169574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b264-1a65-4e94-ab3b-c617396ccfe9 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.172201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 256/530] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:10.173143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230b264-1a65-4e94-ab3b-c617396ccfe9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:10.269785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029993eb-7f45-4aa6-914e-218f8212293e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:10.294666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:10.336120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:10.337269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.338068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.338329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.342660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230b264-1a65-4e94-ab3b-c617396ccfe9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:10.357159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:10.361285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:10.364624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:10.388372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:10.390709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:10.393120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:10.393120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:10.393120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:10.393120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 275/531] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:10.393120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:10.416739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:10.417870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.418460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.419147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.420645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:10.421189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:10.424340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f8bdf-7e02-478c-a991-2e7fbb861679 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.473271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 257/532] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:10.473271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.473271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.474119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.499784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:10.501119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:10.543965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf198-166e-4ca1-b7aa-3b6ecc28cbf6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:10.599708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:10.600769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.601496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.602006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.618495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 258/533] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:10.623386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:10.627892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:10.633421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:10.663397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:10.665096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:10.666279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:10.666279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:10.666279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:10.674661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 276/534] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:10.674661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:10.746946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:10.772664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:10.826946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ff9d0-bcc4-4c56-bb3f-19647dfd8b68 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:10.879174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e40e2-6a55-45df-84c9-7b254e622bcb None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:10.886250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:10.887114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:10.887567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:10.887978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:10.889012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=ccafefb49d0c4727bad188927ae5bf7f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:10.889468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:10.891390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:10.909692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 277/535] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/role_assignments?user.id=ccafefb49d0c4727bad188927ae5bf7f&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:10.909692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.001886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.003421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.004379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.006352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.008113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:11.008770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.013863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.035312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 259/536] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:11.041519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.147535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe375d7-e4f7-4af0-8794-e4fac85a5cd4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.167148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.215420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.216800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.217404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.218041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.219498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:11.220182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.222934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.246810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37567-f1ed-4552-b948-a3da54a77632 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.253282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 278/537] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:11.263479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.296661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.312929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.313580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.313907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.314172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.328766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:11.336548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:11.337794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:11.339160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:11.345852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 260/538] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:11.347478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.450752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc0098-91d3-4587-a192-fd0c506edb0d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.502184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.503364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.503768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.504065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.519375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.521925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:11.545788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:11.546708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:11.547538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:11.548427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 279/539] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:11.549974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.551572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6773-3326-4579-a1ac-2836bd3dc914 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.594938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.595975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.596464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.596978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.598684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.607329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.609181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:11.622865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:11.623775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:11.624609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:11.632210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 261/540] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:11.633692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.684099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.719049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c827e3-e1d4-470f-bf92-8eead87d883f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.752428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b9f35-cb8b-41cf-ba61-a844d937e04a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.767900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.768475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.768797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.769065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.769932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:11.770240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.771979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.791752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 280/541] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:11.792825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.795925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.796978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.797608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.798123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.799089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:11.799653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.801401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.816262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 262/542] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:11.817539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:11.919680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.939413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98123e2-5e23-482d-9039-f0e92f0cd47a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.950518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018531f9-6148-480f-8af1-af599fedb87c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:11.952848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:11.981511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.982186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.982493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.982759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:11.983661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:11.983967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:11.985752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:11.997838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:11.998895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:11.999397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:11.999876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.012912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:12.014834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:12.029042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 281/543] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:12.029042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-60583405-2ae2-4e96-942c-343b03a4055b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:12.033612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:12.035472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 263/544] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7e88-d92d-4b6c-971b-df9aea5b2acc None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7e88-d92d-4b6c-971b-df9aea5b2acc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7e88-d92d-4b6c-971b-df9aea5b2acc None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 264/545] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:12.039975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7e88-d92d-4b6c-971b-df9aea5b2acc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.186990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7e88-d92d-4b6c-971b-df9aea5b2acc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:12.210609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878ca2-88c6-4698-bc44-04ce14fbda1c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:12.230846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:12.231460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.231731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.232013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.232728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=b52624a6d02e43aeb9bb33e61f1bfac0) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:12.233061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:12.234589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166374-b49d-48cf-a43c-79ad40646224 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:12.249949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 265/546] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/role_assignments?user.id=b52624a6d02e43aeb9bb33e61f1bfac0&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:12.249949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.249949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.249949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.275582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:12.276211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.276486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.276755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.277516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:12.277879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:12.279594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:12.291685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:12.293828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:12.294181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 282/547] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:12.295175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.321721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:12.351431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:12.459991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 266/548] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:12.531108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e08b1-f2fd-4fca-9d8f-950f5581cdca None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:12.573102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:12.573752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.574030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.574317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.574960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:12.575258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:12.576667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:12.599864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 283/549] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:12.605603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.803736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147624fe-43ed-4e42-9bac-fa03c0641366 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:12.862028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:12.862721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:12.863037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:12.863483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:12.874788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:12.876686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:12.893952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:12.894937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:12.895866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 267/550] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:12.902066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.904815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:12.924563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:13.054461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002331e-615b-4c2c-b686-5bc2b4c98346 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.095879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:13.096532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.096805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.097096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.112191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:13.122300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:13.129445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:13.132338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:13.134036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 268/551] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:13.140482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:13.172495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11f93f-76a3-4731-a11c-73ad0771f009 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.214622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:13.233235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:13.234622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.235104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.235576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.236936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:13.237616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:13.240609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46057b8-55e4-4a31-bef7-1748e804fe0c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:13.262850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b43cb-6b84-4a99-8167-45fc098233db None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.281790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 284/552] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:13.287480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121aa91-9044-416a-8906-c223c57f1dc1 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.288013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121aa91-9044-416a-8906-c223c57f1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.288590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121aa91-9044-416a-8906-c223c57f1dc1 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.293206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 285/553] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:13.295393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121aa91-9044-416a-8906-c223c57f1dc1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:13.297021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:13.297667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.297946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.298213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.298961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:13.299260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:13.300680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b0904-772a-4500-a0c3-705528b35bc9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:13.310520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 269/554] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:13.311718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.312005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.312273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.337554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:13.338849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:13.417605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 270/555] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:13.418648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:13.498728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:13.513483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121aa91-9044-416a-8906-c223c57f1dc1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.590696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e2ca-b6a4-4598-a499-c70b0c1d4f10 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.603670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:13.613562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:13.614227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.614479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.615309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.706500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:13.707130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:13.707414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:13.707686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:13.739013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:13.741700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:13.784278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:13.787181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:13.789690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:13.791568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:13.793447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:13.793447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:13.793447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:13.797559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 271/556] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:13.807698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:13.967151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968516d3-df39-4e14-8f05-9e9be57f0a4e None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:13.987019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 286/557] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:13.989013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.030245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.042955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.045356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.046909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.048255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.050173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.050779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:14.051103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.052608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.069287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 272/558] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:14.069845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.189410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e4fa5-2a62-49b8-879f-88b6b5a28a05 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.205766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7ca58-e8d2-4d5c-b43f-04018aeeec4b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.231844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.232497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.232674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.232950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.245033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.246569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:14.269460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:14.272079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:14.275366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:14.275366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:14.275366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:14.275366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 273/559] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:14.275366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.310382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.311320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.311798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.312098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.312371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.336616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:14.347656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:14.349854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:14.351398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:14.356584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 287/560] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:14.358313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.406913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4049a3-4dfc-4c86-abba-d863bd6beb80 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.445531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.446314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.446591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.446854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.456000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.457530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:14.470764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:14.472191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 274/561] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:14.473411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.567714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.575808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.594159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4d73-d5da-42f5-acc9-02fbf66bae3d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.595390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446be42-8c58-4144-abbf-0394cf38fd95 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.636518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.636979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.637236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.637488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.638194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:14.638497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.639944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.651867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 275/562] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:14.652634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.681163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.683287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.684107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.684943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.686545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:14.687457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.690447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.718164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 288/563] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:14.720299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.773872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4e43-b691-4873-b739-9af65b805e10 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:14.815209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:14.815866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:14.816148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:14.816414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:14.817164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:14.817461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:14.820021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:14.894392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 276/564] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:14.895951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:14.905970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:14.930016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784667-89e6-43d0-b41a-1c8779d36b1a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.001509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.003047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.004456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.005032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.023560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.025367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:15.043707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:15.044901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:15.046021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:15.046021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:15.046021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:15.052700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 289/565] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:15.053933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.059732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:15.113785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfcef-48e9-4df2-9ff3-da98e5668545 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.160457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.161109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.161534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.161963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.170199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.171717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:15.185543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:15.186930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 277/566] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:15.187824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.242833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57340d-a49f-4004-a6ca-92f740edf257 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.285390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:15.303420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.304248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.304524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.304789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.317389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.319291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-21c04440-46bc-44ff-8f20-7aa43178ed09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:15.338148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:15.348878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:15.351156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:15.351156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:15.351156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:15.351544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1699c-6216-4b54-b572-e67b15a296bf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 290/567] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444206-db7b-4cb1-8c14-337426d314bb None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444206-db7b-4cb1-8c14-337426d314bb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444206-db7b-4cb1-8c14-337426d314bb None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 291/568] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:15.351700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444206-db7b-4cb1-8c14-337426d314bb None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.393271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.393989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.394337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.394764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.395574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:15.396560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.398538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3d53b-f17d-4b64-9685-a519145ce77d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.414879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 278/569] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:15.417086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c77af-edf7-4e31-8e39-39d33c3fa827 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.417559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c77af-edf7-4e31-8e39-39d33c3fa827 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.418168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c77af-edf7-4e31-8e39-39d33c3fa827 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.424577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 279/570] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:15.424577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c77af-edf7-4e31-8e39-39d33c3fa827 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.436328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:15.511801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444206-db7b-4cb1-8c14-337426d314bb None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.559770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.560741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.561189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.561668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.562824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:15.563321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.565904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf9d95-c76c-42ec-bd32-ac1559b04c0f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.582831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c77af-edf7-4e31-8e39-39d33c3fa827 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.586846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 292/571] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:15.588172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a135c-4510-4577-a702-6c3a7805a6df None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.588574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a135c-4510-4577-a702-6c3a7805a6df None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.589013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a135c-4510-4577-a702-6c3a7805a6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.617584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a62a135c-4510-4577-a702-6c3a7805a6df None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:15.619581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a62a135c-4510-4577-a702-6c3a7805a6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:15.626331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:15.641219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.642048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.642446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.642807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.643638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:15.643982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.645581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.662442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 280/572] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:15.664058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.728073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 293/573] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:15.730238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.730680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.731086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.763421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:15.765392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:15.826154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:15.856160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bdea7-18a2-4a92-961a-b0338c3fbdfb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:15.869548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 294/574] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:15.870948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:15.923395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:15.924578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:15.925206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:15.925837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:15.939799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:15.942292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:15.963848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:15.973009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 281/575] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:15.977980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.063474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:16.099842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c885b-afc1-4094-b30b-74d2e650c79e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.141753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735a773-5962-420b-8f51-ab41605b70a8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.146937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.147755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.148180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.148586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.149518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=f7848ff43a794bd4aa35bdff59a53c73) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:16.150764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:16.152465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:16.161527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 295/576] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/role_assignments?user.id=f7848ff43a794bd4aa35bdff59a53c73&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:16.167337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.187780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.188515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.188956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.189350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.199931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:16.201692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:16.215438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:16.217016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:16.218637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:16.221774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 282/577] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:16.222868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.271667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:16.357067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd81b1-aa78-4e21-bcaf-5e55e33edf35 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.360348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:16.403225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b61f06-4656-4c1a-bc6d-fbd0e89b5839 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.403844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.404585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.404923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.405253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.429084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:16.435453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:16.437489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 283/578] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:16.439630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.471516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.472399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.472804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.473198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.474863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:16.475346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:16.477416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:16.500366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 296/579] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:16.501846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.656472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4338-d6d1-44f5-8bb6-7cf1b454d035 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.664077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:16.668326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2d08-03f8-4a96-a54e-36fa41116bcb None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.720064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.720578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.720866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.721221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.722039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:16.722344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:16.723921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:16.729670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.731794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.732603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.733976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.741332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 284/580] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:16.742897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.758515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:16.767309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:16.769069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 297/581] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:16.775173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:16.798651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:16.915037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70388dea-1a53-443b-b80f-79f324a1dd76 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.965563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ad3f-81cf-46bd-8faa-2aacfe323080 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:16.982905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:16.983881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:16.984352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:16.984825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:16.986066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:16.986437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:16.988427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.006244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 285/582] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:17.007512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.028877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.030741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.031215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.031628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.048535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.050414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:17.071481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:17.072623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:17.073622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:17.073622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.073622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:17.075647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 298/583] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:17.076649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.097348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:17.183173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:17.207106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d526db-5d41-4f4d-99c7-42cf20d4dd69 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.253850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4a75-e7e9-44b7-a73b-29409144539d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.270856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.272475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.272938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.273411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/7739336ec72248feb11d1d6a7a8de166/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.298408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=7739336ec72248feb11d1d6a7a8de166, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.299929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.311462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.312520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.312999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.313485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.314779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:17.315305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.318416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.330006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 286/584] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:17 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/7739336ec72248feb11d1d6a7a8de166/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:17.330006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.336427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 299/585] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:17.337292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.484762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:17.553927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21225c-68e9-4ca0-b0e6-ead96b06a02b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.572442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29104f4f-2b83-4c47-95f4-9948ae0e7ae1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.617723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.618667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.618951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.619220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.619902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:17.620205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.622149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.638507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.639929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.640621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.641172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 300/586] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:17.641550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.649679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.655885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:17.675917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.680421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:17.712194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:17.714278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:17.716018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:17.716018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.716018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:17.718926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 287/587] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:17.720661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.785214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a890d4-52c3-40cb-9cbd-dee4231b429e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.830088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:17.838421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.839135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.839412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.839706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.854248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.856374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:17.877097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:17.879206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:17.881334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:17.881334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:17.881334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:17.881334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 301/588] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:17.881334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:17.910267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e6981-988f-4639-9231-cf5e82dfd4bb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:17.956014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:17.956927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:17.957512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:17.958040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:17.966787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:17.968546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:17.981983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:17.983086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:17.984126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:17.990051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 288/589] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:17.991292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.040357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a80603-680f-47a0-99ff-6c2fbc8088f8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.057634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.085404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.086355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.086789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.087225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.088076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:18.088538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.090218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.105163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 302/590] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:18.106462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.142002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638042-d397-4016-910d-a76c2672af4d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.169087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.202034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.202967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.203402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.203848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.204678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:18.205141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.207591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.228963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 289/591] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:18.230121 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.246797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527c8d-d1cb-4bb6-9292-30de83184d15 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.310818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.312489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.313254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.313718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.314908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:18.315388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.318135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.342031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 303/592] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:18.343026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.370865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b161bf-3fa5-4098-a466-cbf3316b4174 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.382438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.413919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.414686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.414965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.415235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.423673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.425303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:18.439315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:18.440337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:18.441278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:18.445714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 290/593] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:18.446732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.482037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b7f5-87fc-44d1-890b-7d4726693d96 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.485412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.525828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.527048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.527547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.528102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.538311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.540373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:18.555102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:18.556119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:18.557059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:18.559831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 304/594] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:18.560986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.577232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683e820-2b71-4688-8a57-d475cf286cc5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.620523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.621122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.621401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.621734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/ccafefb49d0c4727bad188927ae5bf7f/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.640631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=ccafefb49d0c4727bad188927ae5bf7f, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.642182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.658059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 291/595] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:18 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/ccafefb49d0c4727bad188927ae5bf7f/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:18.658721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.690706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.714761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaaf23-e9d9-42cf-a339-94bdcf3499e3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.776369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.777444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.778032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.778673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.780156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:18.780758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.784372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61f464-e81a-48d7-bbe9-d09201f3c39e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.826366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 305/596] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:18.829231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ade07-2b56-4cfe-b519-36e65be48b97 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.829789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ade07-2b56-4cfe-b519-36e65be48b97 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.830285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ade07-2b56-4cfe-b519-36e65be48b97 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.838300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 306/597] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:18.840288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ade07-2b56-4cfe-b519-36e65be48b97 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:18.843300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc345-b1fe-4129-ae3b-b54efc168548 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:18.905128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:18.919053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:18.920077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.920481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.920885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.922147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:18.922649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:18.925239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc54e7-af27-4d95-9314-d8213a972a26 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:18.957207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 292/598] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:18.963667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:18.964151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:18.964615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:18.995302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:18.997333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:19.044797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ade07-2b56-4cfe-b519-36e65be48b97 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:19.117885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:19.118836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:19.119355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:19.119797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:19.126216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 293/599] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:19.127776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:19.150547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:19.151165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:19.154866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:19.214202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:19.215336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:19.216223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:19.216223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:19.216223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:19.217014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 307/600] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:19.227150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:19.351864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041cfaa-66a2-4958-8c95-93d4f9281cc7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:19.423672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:19.426719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:19.426719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:19.426719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:19.434691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:19.448429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:19.451724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:19.485513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:19.486551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:19.487486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:19.487486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:19.487486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:19.492062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 294/601] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:19.495392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:19.534001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df0f1b-a69d-4a0e-976d-8ccac0c0af1a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:19.690070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:19.723143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:19.724842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:19.725429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:19.725942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:19.726053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b16a-f568-4a6c-a7a2-70b73154f3b0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:19.727070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:19.727516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:19.738725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:19.783624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 308/602] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 558 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:19.798635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:19.830646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:19.831709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:19.832208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:19.832714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:19.835197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:19.835874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:19.838560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:19.883408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 295/603] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:19.885593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.027832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08a6b6-00e5-4d04-89b6-8bea4cecea7b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.028475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2149c-26b9-47c6-a420-ac57e52a719f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.048544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.090089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.090402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.090742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.091020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.091289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.091425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.091892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.092382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.097047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.104863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:20.108296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-6107bb0c-f1c2-4bb4-a18f-820e0036c302 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:20.110168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:20.111112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:20.114686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:20.115843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.115940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 296/604] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:20.115940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4c3bc-013d-4419-88b4-58df7b485c37 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.115940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4c3bc-013d-4419-88b4-58df7b485c37 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.115940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4c3bc-013d-4419-88b4-58df7b485c37 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.119659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 297/605] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:20.120726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4c3bc-013d-4419-88b4-58df7b485c37 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:20.131332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:20.132294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:20.133188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:20.136460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 309/606] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:20.137566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.285574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4c3bc-013d-4419-88b4-58df7b485c37 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.291887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8305-ff08-41d7-a45c-b206954e49b4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.337382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.338037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.338315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.338588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.339256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:20.339557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.340971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92f00-9885-49f7-a335-60b8823f56cd None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.355501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 310/607] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:20.355501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.355501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.355501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.356670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.357534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.358090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.358589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.372758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.375631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.376529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:20.377927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:20.399971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:20.401833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:20.403768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:20.406432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 298/608] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:20.407417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.464894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.465525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.487909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 311/609] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:20.490255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.574096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779429e-8fab-41ef-a107-069889d5ce9f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.634271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.635589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.636050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.636515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.637935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:20.638460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.641241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.644585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf985c5-4289-456f-ace2-d550354dc994 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.664353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 299/610] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:20.666039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.716144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.717271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.717810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.718298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.739370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.742426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:20.774652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:20.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:20.776559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:20.776559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:20.776559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:20.784492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 312/611] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:20.786149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.856412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.869414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:20.875394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd7ec-7bbb-4a54-a541-d71b9ac83c74 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.938013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.939047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.939499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.939955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:20.940847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7991f46-0c8d-41f5-95fd-8c72dec4fa93 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:20.941236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:20.941807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:20.945200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:20.975650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 300/612] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:20.975650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:20.992838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:20.994112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:20.994112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:20.994410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/b52624a6d02e43aeb9bb33e61f1bfac0/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.013556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=b52624a6d02e43aeb9bb33e61f1bfac0, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.015069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.031030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 313/613] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:20 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/b52624a6d02e43aeb9bb33e61f1bfac0/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:21.032007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.152278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e21ee-25d0-471f-8625-2da9bac08eb5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.167826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:21.190834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29a48-a081-491b-989e-7a011c556401 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.204349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.205004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.205267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.206255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.206976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:21.207286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.209291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.233679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 301/614] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:21.233679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.255364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.256781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.257057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.257328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.270903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:21.277631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c76abd48-54b8-421d-9293-18c7d8e879e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:21.284701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:21.286860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 314/615] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:21.289554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a6df-8868-4d09-a626-92fe50da305b None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.291297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a6df-8868-4d09-a626-92fe50da305b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.291297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23a6df-8868-4d09-a626-92fe50da305b None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.291297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 315/616] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:21.291297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23a6df-8868-4d09-a626-92fe50da305b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.393514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17382e-5cbc-4f5e-964e-de6302bd7df1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.450752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.451813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.452253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.452692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.480877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=9260fd9f9c524d20a9025595a53ade4e, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.483358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.483718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23a6df-8868-4d09-a626-92fe50da305b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.501581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 302/617] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:21 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:21.503515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.563305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.564304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.564719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.565142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.580840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.582951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-7bed34c6-4b83-4c61-a8b6-aec71aa4eb61 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:21.608889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:21.610006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:21.610957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:21.611653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 316/618] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:21.614316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.614877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.615930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.616861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:21.654232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:21.657464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:21.663672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:21.671081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5c8c2-a005-4525-960d-5f2eeb6aca78 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.725966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.726542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.726824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.727096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.727852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:21.728107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.729542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.743504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 303/619] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:21.743757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.793819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 317/620] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:21.794770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.870583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7104d-a5b1-4e7d-9596-49bec14cb98d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:21.905285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:21.912416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:21.913200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:21.913639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:21.914157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:21.915006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:21.915320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:21.916853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:21.939364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 304/621] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:21.945354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:21.989039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbec465-f2c2-435f-bdee-f09f3b8b9a6c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.069812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe704-3ace-4fdb-95f2-7aa3dd49c63c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.113274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.114072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.114356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.114631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.127839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.129347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.147492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:22.148352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:22.151613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:22.155094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:22.155094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:22.155094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:22.155902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.155902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.155902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.156584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 305/622] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:22.156584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.170956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.182634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.186612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:22.215271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:22.218991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:22.222347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:22.222347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:22.222347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:22.222347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 318/623] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:22.222347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.234479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.310228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f671-54a5-468b-86ae-dfdb05b8e243 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.351881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221cad8-acb5-4d8c-a0e2-3098d4ab64dc None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.353750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.354426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.354706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.355088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.355799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:22.356203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.357776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.372635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 306/624] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:22.373714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.403033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.403793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.404077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.404444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.418308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:22.423956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:22.426228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:22.428386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:22.433769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 319/625] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:22.433769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.495665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.499140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d6152-b44b-4dbc-925d-2d7ffc20f8f7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.506999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.543254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.544297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.544785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.545242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.546226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:22.546722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.548526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.563914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 307/626] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:22.565280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.569730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558aaf-4c7d-4155-bea0-2bc0c010dbe5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.614378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.615704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.616147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.616431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.625633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.627283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:22.641861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:22.642870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 320/627] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:22.644189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.697327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797146a2-3950-447e-83b8-f309a309a481 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.740777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.741878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.742444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.742981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.744080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:22.744603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.746393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.760242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 308/628] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:22.761842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.785183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0fcad-4193-4842-8b84-8ce3dbf7aa17 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.794140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.797794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:22.828830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43186afc-1533-4560-a7da-64a55224523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.829499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43186afc-1533-4560-a7da-64a55224523d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.830037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43186afc-1533-4560-a7da-64a55224523d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.830318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43186afc-1533-4560-a7da-64a55224523d None admin] PATH_INFO: `/v3/users/glance` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.840950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43186afc-1533-4560-a7da-64a55224523d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.842573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43186afc-1533-4560-a7da-64a55224523d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-43186afc-1533-4560-a7da-64a55224523d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:22.857121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:22.858441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 321/629] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:22.859637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43186afc-1533-4560-a7da-64a55224523d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:22.917725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ca0-1fe2-4b73-9505-112a9f2497b9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:22.983538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:22.984294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:22.984665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:22.985194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:22.986861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:22.987531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:22.990991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.022080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 309/630] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:23.023368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43186afc-1533-4560-a7da-64a55224523d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.023782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.069216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.070103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.070389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.070656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.084466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.086455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:23.104252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:23.107819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:23.111213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:23.111213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:23.111213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:23.111213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 322/631] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:23.111213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.126455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.145274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.149876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e2303-a49e-48e4-87e0-29f11c73a9e4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.189075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.189914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.190290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.190661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.191504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:23.191910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.194596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.220473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 310/632] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:23.221667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.244753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8356-0be2-4734-8dda-48847a88fa1e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.293324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.294313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.294881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.295303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/f7848ff43a794bd4aa35bdff59a53c73/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.314257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=f7848ff43a794bd4aa35bdff59a53c73, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.315982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.337970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 323/633] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:23 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/f7848ff43a794bd4aa35bdff59a53c73/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:23.338943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.361617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbd092-9572-478c-abde-f5830de76798 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.435231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.436036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.436405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.436773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.437533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:23.437964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.439495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.446058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.465729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.474478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 311/634] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:23.476347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb8a24-7de1-4ce6-8e0f-fc7d70ae4839 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.488152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.586470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.587379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.587848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.588334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.681454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602956e-9568-4b49-91fd-e4a0ea852b32 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.689741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.691467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa64b22-ca85-46e4-9da8-7e0be3b2f9fa None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.729060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.730161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.730632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.731090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.740731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.742760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-a8001870-45f6-4a2d-9b4c-d8c38b0b0a9a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:23.760718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:23.761976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:36:23.763014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:23.763839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 312/635] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:23.765061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ec12-e22f-451e-81dc-8ade5d4b87fc None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.765300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ec12-e22f-451e-81dc-8ade5d4b87fc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.765597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ec12-e22f-451e-81dc-8ade5d4b87fc None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.775784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 313/636] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:23.775784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ec12-e22f-451e-81dc-8ade5d4b87fc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.801189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.852008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 324/637] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:23.854001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.854288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.854562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.866340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:23.882824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:23.884420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:23.925603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ec12-e22f-451e-81dc-8ade5d4b87fc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:23.969152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 325/638] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:23.970083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:23.970405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:23.971061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:23.971496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:23.971919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:23.972855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:23.973431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:23.975937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:23.998272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 314/639] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:24.004263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.145028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2959-6a29-499f-8219-18756f221e9b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.167131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.179214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a41d8-8866-4c12-a5ea-ec59d48db5e0 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.181044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.189554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.190600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.191092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.191585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.253567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.254253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.254532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.254882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.280755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.282663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.295257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.298485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:24.339999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:24.341043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:24.341993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:24.341993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:24.341993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:24.341993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 326/640] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:24.345387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.408943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 315/641] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 11:36:24.414746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.437482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.501351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836e615-c76d-4f59-8981-0a132d064467 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.551369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.552032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.552541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.552859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.553507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d052e-729a-4f86-81b1-ac6ecaa52d9f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.568386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:24.575848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:24.576944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 327/642] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:24.582967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.599832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.607878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.608928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.609514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.610038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.610975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:24.611489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.613408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.628467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 316/643] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:24.629703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.717493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ad17-126c-4f32-8a74-75180d57878b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.733778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.756987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.757606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.757953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.758220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.765069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6468c-5f18-42b9-b351-de22f2f1fb2c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.770654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.772616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:24.789984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:24.791320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:24.792369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:24.792369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:24.792369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:24.792369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 328/644] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:24.792369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.815697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.816839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.817378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.818587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.819692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:24.820219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.822022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 317/645] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:24.839560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:24.870125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:24.911102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7fcc3-d77b-4f37-a471-c46adf4d7624 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.949274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:24.949946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:24.950205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:24.950473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:24.958085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:24.960278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:24.972634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:24.973574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:24.975019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:24.977763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12b10-ecd0-4ab6-b029-2d6f01f7fdcb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:24.978937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 329/646] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:24.979889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.025575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.026242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.026657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.026904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.027697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:25.028460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.029509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.040999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:25.050360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 318/647] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:25.051499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.105258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a349-45cd-4c51-8b86-12fd976061aa None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.153896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:25.176055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.177090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.177689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.178137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.201155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.204528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.205076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43677f-506a-42f1-9761-f7b61f377836 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-89a173f6-0258-4ce3-9758-05a523a8c36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:25.235581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:25.236683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:25.237744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:25.237744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:25.237744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:25.248193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 330/648] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:25.248683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5bae2-cff8-4044-9370-758992fb9835 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.249156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5bae2-cff8-4044-9370-758992fb9835 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.249622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5bae2-cff8-4044-9370-758992fb9835 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.259337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 331/649] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:25.259337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5bae2-cff8-4044-9370-758992fb9835 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.270374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.271418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.271418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.271937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.272293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:25.272598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.274975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.300873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 319/650] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:25.300873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.406511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:25.469437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5bae2-cff8-4044-9370-758992fb9835 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.500763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0468820-999f-4436-b6a2-455fc7f11159 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.534839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.535931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.536387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.537376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.549463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.552029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.567243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.568857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.569324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.569834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-940b638c-7910-4f61-ac90-9341a318f295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:25.570926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:25.572529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:25.574284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:25.576501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:25.576501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.576501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.577669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 332/651] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:25.581257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.581738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.582147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.597046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 320/652] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:25.597756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.615792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:25.617755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:25.680813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:25.738300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a94ef-f390-4975-88d5-32cf6ca90f97 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.758544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 333/653] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:25.760015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.778744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:25.798612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.799244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.799519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.799822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.800538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=7739336ec72248feb11d1d6a7a8de166) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:25.800966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.803225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:25.810998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 321/654] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/role_assignments?user.id=7739336ec72248feb11d1d6a7a8de166&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:25.811641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:25.922436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fa7a1-695a-4921-bc65-f0a2b71b839a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.986409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:25.988294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:25.989613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:25.991404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:25.993266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:25.994920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:25.997253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8ea7-1e17-49c2-b68d-b46da71e406a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:25.999504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aeb58-fdc0-45a6-a150-e3ded131bc6b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:26.033788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 334/655] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:26.036537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8bc3-29dd-45a7-a433-353657d89d84 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.037524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8bc3-29dd-45a7-a433-353657d89d84 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.038054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8bc3-29dd-45a7-a433-353657d89d84 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.041853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 335/656] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:26.043660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb8bc3-29dd-45a7-a433-353657d89d84 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.050300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.051990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.052631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.052631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.064467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:26.067394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:26.083209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:26.085809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 322/657] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:26.087767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.107281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:26.157395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:26.235349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb8bc3-29dd-45a7-a433-353657d89d84 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.255291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65de9-170a-4943-80d0-54feca6dde26 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.288865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.289891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.290374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.290721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.311113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:26.317109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.318659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.319084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.319480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.320386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=ccafefb49d0c4727bad188927ae5bf7f) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:26.320822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-652e71c4-d7b3-4b29-af6e-ab2f0dcffd76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:26.322011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:26.330376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:26.332437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:26.335751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:26.335981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 336/658] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:26.335981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.335981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.335981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.340691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 323/659] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/role_assignments?user.id=ccafefb49d0c4727bad188927ae5bf7f&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:26.341571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.364047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:26.368669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:26.513684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 337/660] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:26.513684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.572500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc22a5-c5b3-4f0a-aae0-30df839b6c2b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.578020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:26.587662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:26.633531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.635374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.635374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.635374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.635915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:26.636293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:26.637951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:26.665762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 324/661] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:26.674783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.686369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a989-6edb-48ef-8429-e85c7d5f2a2a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.735468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.736519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.737031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.737582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.738556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:26.739105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:26.741014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:26.752622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 338/662] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:26.761218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:26.870626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee0725-ea5e-43d7-83a7-35b27f116572 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.903669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:26.923508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:26.924524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:26.925012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:26.925487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:26.940386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d08268-a435-43e2-a28e-e422f556eaa2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:26.967132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.003566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.004671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.005963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.007129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.008618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.011048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.031768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.034669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:27.063072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:27.064117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:27.065101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:27.065101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:27.065101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:27.070063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 339/663] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:27.071799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.154394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 325/664] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:27.156009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.222746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318b1c-c5a4-45b5-90d8-825bbeaccf02 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.266254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.268806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.269810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.270002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.270254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.284339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:27.290101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:27.292515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 340/665] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:27.295336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.303393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.379116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae165c-38dc-4b26-9907-d5893fea1d66 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.413506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615893-34de-4ce6-90b8-5812bac65f2a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.446314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.447834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.448566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.449312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.450954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:27.451760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.454008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305801-0647-454e-b162-feadd4741943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.454714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.454954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305801-0647-454e-b162-feadd4741943 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.455342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305801-0647-454e-b162-feadd4741943 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.455719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305801-0647-454e-b162-feadd4741943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.468599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305801-0647-454e-b162-feadd4741943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.470325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305801-0647-454e-b162-feadd4741943 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.485463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 326/666] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:27.487577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-6a305801-0647-454e-b162-feadd4741943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:27.487812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:27.489975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:27.491815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:27.491815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:27.491815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:27.495834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 341/667] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:27.496904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305801-0647-454e-b162-feadd4741943 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.551377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.618549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305801-0647-454e-b162-feadd4741943 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.658980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.659674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.659951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.660225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.670281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.671691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:27.685892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:27.686901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:27.688270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 342/668] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:27.689226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.700670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ec5a-5f8c-4961-80af-966d4fd5267b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.754598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.770400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.771512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.773163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.773163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.774340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:27.774863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.777430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.799638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 327/669] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:27.801210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:27.834992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16355e-0392-4c7f-a382-8567ae8faf5f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:27.842227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:27.890539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:27.891212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:27.891492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:27.891817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:27.892488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:27.892792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:27.894266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:27.911299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 343/670] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:27.912318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.024190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529cb37-4f79-45cf-b981-46a22ed1962a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.051606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02423-20d6-4c0a-8d44-10fa2a45a303 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.066477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.067283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.067596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.068013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.068981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:28.069343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.071082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.094285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 328/671] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:28.095042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.098233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.099184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.100981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.101369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.116923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.118734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:28.136475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:28.139301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:28.147554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:28.147554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:28.147554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:28.147554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 344/672] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:28.147554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.160073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.168830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.223784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c993-8278-48d3-b054-52e9d8d62f3c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.275273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.276938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.278036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.278576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.284751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16a64-fcd1-4c1b-a169-4c960ba680e9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.288355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.290345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:28.305074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:28.306241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:28.307150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:28.309729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 329/673] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:28.310935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.337881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.338602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.338796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.339067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.429051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.430760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.436474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9865d-1582-4ceb-b7a6-0552eeba10f7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.446356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.455985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.481897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.482622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.482975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.483324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.484077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:28.484459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.486292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.504752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 330/674] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:28.506802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.564411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 345/675] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:28.565688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.630487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52d9a-f6fe-488b-821b-4cd8d3de3b2f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.672964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.673787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.674197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.674476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.675346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:28.675777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.678187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.697697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 331/676] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:28.697697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.705170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071235-f858-41fc-bea7-805d5e0a6373 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.727124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.733225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:28.751896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.752551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.752831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.753102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] PATH_INFO: `/v3/users/swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.762438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.764020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:28.778754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:28.779765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:28.780719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:28.784649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 346/677] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:28.785573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.823967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7281d7e-ded4-414f-98ff-e9a8f88a47f5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.868004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.868656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.868934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.869225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.882513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.884111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4cf4a802-a9db-4a1a-910a-57c3591be64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:28.906854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:28.907924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:28.908816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:28.908816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:28.908816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:28.908816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 332/678] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:28.910254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa6d7-6ced-44d2-983b-bdf1c80d69d8 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.910606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa6d7-6ced-44d2-983b-bdf1c80d69d8 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.910887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa6d7-6ced-44d2-983b-bdf1c80d69d8 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.912945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 333/679] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:28.913271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa6d7-6ced-44d2-983b-bdf1c80d69d8 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:28.932059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946486a2-4cc0-4d3e-870e-569fc945ebef None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:28.978293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:28.978955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:28.979273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:28.979483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:28.980238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=b52624a6d02e43aeb9bb33e61f1bfac0) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:28.980541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:28.982047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:28.989547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 347/680] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/role_assignments?user.id=b52624a6d02e43aeb9bb33e61f1bfac0&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:28.990459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:29.043494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.062094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.079453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa6d7-6ced-44d2-983b-bdf1c80d69d8 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:29.147274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:29.148350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.148838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.149302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.150900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:29.151511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:29.154316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229039-42d9-4656-a17b-7e9c5b62de30 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:29.155314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e572f0-9a53-4d8e-ace9-3c91b006b5db None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:29.186462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 334/681] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:29.187363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.187764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.188052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.211755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:29.213723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:29.217177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:29.218065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.218520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.219841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.220118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:29.220549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:29.222692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3dab-8234-4c55-84d7-57a57efa2608 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:29.247228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 348/682] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:29.249406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adafb4-6402-4c7f-957a-715083619566 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.249872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adafb4-6402-4c7f-957a-715083619566 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.250347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adafb4-6402-4c7f-957a-715083619566 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.252668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 349/683] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:29.254935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adafb4-6402-4c7f-957a-715083619566 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:29.336618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 335/684] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:29.337882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:29.416372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.421775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adafb4-6402-4c7f-957a-715083619566 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:29.453446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.491752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:29.493914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.494731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.495442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.510952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95442e-9b7c-4209-b29c-18b0ef2587c9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:29.580532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:29.581959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.582601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.583244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.597504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:29.600385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f4ec226b-d4d8-4754-9aae-e8e70d406087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:29.617233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:29.618323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:29.619205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:29.626424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 336/685] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:29.628877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:29.629617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:29.630397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:29.636235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:29.638737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:29.657993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:29.660256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:29.777879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 337/686] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:29.778826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:29.848828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.862181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 350/687] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:29.871132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:29.881237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:29.985090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a41f8-f7f3-441f-8089-feb0d705ea9d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.030659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.031287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.031577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.031823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.032546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:30.032852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.034454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.052880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 338/688] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:30.054103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.080043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd9d-dde1-4f33-ad68-ff0dea05a909 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.126294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.126848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.127149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.127401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.141148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.142736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:30.161953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:30.163511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:30.165046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:30.165046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:30.165046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:30.168347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 351/689] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:30.169364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.226139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee9b1-710a-47c6-9da9-36943baada7a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.246384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:30.258102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:30.273148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.274208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.274769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.275231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.276062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.277843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:30.291054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:30.292148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:30.292985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:30.300753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 339/690] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:30.302099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.330890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23ea4-ca9f-4320-8a1b-0c1b5bcd6712 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.388371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.389297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.389811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.390284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.391654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:30.392292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.395231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.407802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 352/691] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:30.409026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.479771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474369-699c-4dfd-a5b8-2dc95f5510b4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.545836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.546916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.547451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.547893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.549033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:30.549523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.552594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.571517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 340/692] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:30.572837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.594520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efebf3-fa63-4e0f-b8bc-0219340e70c2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.632847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:30.647117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:30.663469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.664095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.664370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.664635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.665283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:30.665585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.667146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.674118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 353/693] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:30.675346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.772349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68474bf-6d8f-42f5-ae9a-dc365adb4b4c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.838576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.840438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.840438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.841144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.873422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.876465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:30.900254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bf64-130e-4478-8fa4-6a00a5b76577 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:30.906507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:30.908328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:30.910083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:30.910083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:30.910083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:30.913956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 341/694] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:30.915680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:30.971975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:30.973187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:30.973740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:30.974731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:30.989507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:30.993129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:31.018042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:31.019994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 354/695] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:31.023035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:31.110763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:31.119865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:31.139671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47d69-e283-487d-8a4f-e653137339fc None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:31.210368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:31.211911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.212668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.213412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.214885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:31.215709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:31.218626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.231778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 342/696] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:31.233991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:31.246626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b15e4-e265-43cd-8691-44d0a2b1ce40 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:31.323983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:31.325217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.325859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.326414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.327763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:31.328501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:31.331364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.360414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 355/697] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:31.367476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:31.506626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a1a2-57d9-480c-829c-5cc20d120831 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:31.578408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:31.579080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.579355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.579628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.580354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:31.580663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:31.582466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.597625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 343/698] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:31.608195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:31.649728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:31.660791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:31.670100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390011a-142a-46be-9fea-42ef30655d6f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:31.737488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:31.738233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.738513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.738788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.740711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:31.742099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75474-a178-48f2-8c94-9d22a7c5cc84 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.782241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 356/699] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:36:31 2022] POST /identity/v3/users => generated 254 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:31.785604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443465b8-efc9-4d20-bef3-520ccd1c8b48 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.786171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443465b8-efc9-4d20-bef3-520ccd1c8b48 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.788053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443465b8-efc9-4d20-bef3-520ccd1c8b48 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.799411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 357/700] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:31 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:31.813803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.814328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.814786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.844111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321c7ba-ac0a-41f4-8be4-9cc363b76a45 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:31.875821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:31.878371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:31.916857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:31.919091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:31.919865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:31.920542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:31.922107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=f7848ff43a794bd4aa35bdff59a53c73) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:31.922876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:31.926318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:31.940107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 344/701] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:31 2022] GET /identity/v3/role_assignments?user.id=f7848ff43a794bd4aa35bdff59a53c73&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:32.074362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 358/702] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 06 11:36:32.486192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:32.588704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:32.611735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baba1f4-8066-45ca-80fb-c72a55e074ac None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:32.656576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:32.657236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:32.657515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:32.657834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] PATH_INFO: `/v3/services/network` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:32.658530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:32.659977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:32.671606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:32.672672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:32.673519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 345/703] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:32 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:32.682499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:32.814828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:32.838298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62163-0bc1-43ad-a984-658cbd50b089 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:32.900621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:32.901322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:32.901637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:32.901932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:32.902602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] RBAC: Adding query filter params (name=network) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:32.902908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:32.904355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55048f-f833-442b-bd90-f51649f7c4cc None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:32.915117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 359/704] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:32.921021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:33.071006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a74f4-bc08-4502-8d46-c6c3f9988fd4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:33.145184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:33.146865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:33.148062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:33.148527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:33.149736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] RBAC: Adding query filter params (type=network) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:33.150499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:33.153106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8fd5f-3261-43ef-98df-33ba45bb400d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:33.170630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 346/705] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:34.131102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb52a7d-aa01-44f8-8e43-8ccdf32231b6 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.131102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb52a7d-aa01-44f8-8e43-8ccdf32231b6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.131102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb52a7d-aa01-44f8-8e43-8ccdf32231b6 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.132555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 360/706] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:34.146636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a98c-8c75-4a82-b041-82405ea3da95 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.147286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a98c-8c75-4a82-b041-82405ea3da95 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.147286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0a98c-8c75-4a82-b041-82405ea3da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.174988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-31c0a98c-8c75-4a82-b041-82405ea3da95 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:34.176346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-31c0a98c-8c75-4a82-b041-82405ea3da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:34.179149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:34.274626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 347/707] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:34.605129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f7da-d621-43d4-869d-f82aa0217094 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.605401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f7da-d621-43d4-869d-f82aa0217094 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.605844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f7da-d621-43d4-869d-f82aa0217094 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.607307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 361/708] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:34.613259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed02f-123a-40b5-966d-d0ba657fcaaa None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.613913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed02f-123a-40b5-966d-d0ba657fcaaa None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.614412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed02f-123a-40b5-966d-d0ba657fcaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.635703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bb9ed02f-123a-40b5-966d-d0ba657fcaaa None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:34.637397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bb9ed02f-123a-40b5-966d-d0ba657fcaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:34.742386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 348/709] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 11:36:34.771691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8188-ece4-4e1b-8d80-516233c40b09 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.772822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8188-ece4-4e1b-8d80-516233c40b09 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.773573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8188-ece4-4e1b-8d80-516233c40b09 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.778524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 362/710] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:34.786092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:34.787110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:34.787879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:34.829823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f8188-ece4-4e1b-8d80-516233c40b09 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:34.839140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:34.839140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:34.998114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:35.017832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 349/711] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:35.042234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.063562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f8188-ece4-4e1b-8d80-516233c40b09 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:35.103611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:35.104265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.104542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.104812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.117327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:35.123336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:35.124476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:35.125528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:35.127820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 363/712] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:35.160273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.178759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:35.301187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133ed4-20e4-4854-a555-ff507627621f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:35.339818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6c38-f464-45cd-8319-1fa438b9b9f2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:35.389297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:35.389984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.390259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.390530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.415198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:35.421348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:35.422514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:35.423422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:35.426414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 350/713] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:35.432442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.437398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:35.438479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.438753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.439194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.440082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:35.440672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:35.442183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:35.452963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 364/714] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:35.454247 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.555153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:35.582350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:35.632289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0ef2-278d-47f4-be30-e7fae4290306 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:35.676157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ad09-ab32-4633-8544-c042bb287d31 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:35.708255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:35.709780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.710575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.710972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.711868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:35.712312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:35.714120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:35.732874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 365/715] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:35.732874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.751425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:35.752425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.752854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.753258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.754163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:35.757290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8fb6c-390c-48e8-aa2d-55cb9645cda2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:35.789996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 351/716] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:36:35 2022] POST /identity/v3/services => generated 251 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:36:35.792130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784b11-023e-4f8e-a46a-138cff312d4c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:35.792418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784b11-023e-4f8e-a46a-138cff312d4c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:35.792918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784b11-023e-4f8e-a46a-138cff312d4c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:35.795261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 352/717] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:35.796309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5784b11-023e-4f8e-a46a-138cff312d4c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:35.956137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a00d-1107-4ec4-ab78-b93fc2781d38 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.026570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:36.051321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.052869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.054193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.054668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.057326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5784b11-023e-4f8e-a46a-138cff312d4c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.074426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.076991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.103102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-540427f9-c3a5-4640-8cea-039d7c8ccbec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:36.104273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:36.106250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:36.108219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:36.108219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:36.108219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:36.110197 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 366/718] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:36.116284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.116893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.118447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.149563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.151036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.151717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.152396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.164404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:36.166997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:36.182861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.187534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:36.224080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:36.226435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:36.228554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:36.228554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:36.228554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:36.232206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 353/719] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:36.234252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:36.342431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 367/720] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:36.345085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:36.401534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01967ae2-fa56-484a-ab9b-66c1fbdcbf12 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.450366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.451868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.452598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.453947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.454936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:36.455462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.457232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.473871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 354/721] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:36.481478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:36.534445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:36.558971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7edbb7-7a71-4d05-926b-03427beb590a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.576424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:36.606390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.607364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.607797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.608220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.609245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:36.609736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.611850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eed2d-2cb5-4ed0-80b4-671616006988 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.631992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 368/722] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:36.639235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052288fe-ccfa-4c86-a81d-9a907f03bdd7 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.640743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052288fe-ccfa-4c86-a81d-9a907f03bdd7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.641178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052288fe-ccfa-4c86-a81d-9a907f03bdd7 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.644747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 369/723] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:36.647292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052288fe-ccfa-4c86-a81d-9a907f03bdd7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:36.680321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac62979-2e96-4507-a699-4170d8dda387 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.727181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.728251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.728746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.729346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.744865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.747817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c47d261d-2d84-448a-8171-c41cc3bbcc50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:36.773343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:36.774438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:36.775488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:36.776029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 355/724] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:36.778029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.778518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.779011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.807435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052288fe-ccfa-4c86-a81d-9a907f03bdd7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:36.813685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:36.815671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:36.867849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:36.868771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:36.869153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:36.869421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:36.879372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:36.882264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:36.896397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:36.898308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 370/725] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:36.901407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:36.951996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:36.962265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 356/726] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 11:36:36.962265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:37.004235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:37.095990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f482e-248d-4405-a059-1306d57d8449 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.163388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.164546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.165405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.165690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.182142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557dc8b-7396-47cc-b7d5-6b9b5a6492ef None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.188941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:37.194992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:37.196473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:37.197449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:37.198702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 371/727] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:37.198808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:37.284873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.285884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.287152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.287651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.289676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:37.290254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:37.292256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:37.328757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 357/728] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:37.334856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:37.406591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:37.420273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e3b7-e084-4a7b-97f0-b2af420d4845 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.477894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.479436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.479934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.480414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.481873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:37.482410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:37.485286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:37.487200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:37.526006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 372/729] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:36:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:37.527507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:37.539118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a86770-991d-4e7d-9e4c-8a60f91fc7dd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.608336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.609401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.609937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.610420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.611765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:37.612316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:37.615681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:37.633946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 358/730] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:37.635726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:37.710340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3f54c-fceb-40ae-ba95-dcdce8637cac None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.754328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.755351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.756025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.756623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.796290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:37.807268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e77b04-90ce-40ce-ae1c-252ec92219ed None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:37.875878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:37.876544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:37.876876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:37.877417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:37.878645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:37.881395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:37.889278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:37.904622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:37.907521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:37.925838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 359/731] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:36:37 2022] POST /identity/v3/services => generated 240 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:37.925838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.053811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 373/732] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 11:36:38.054835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.122285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:38.157086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7381-2c24-4285-bfa2-25e93850af3f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.187080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d89c-7549-44de-a41e-faf38b43cd9d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.222569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.223221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.223500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.223776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.268185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.269262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.269854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.270382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.283772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.285590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:38.303617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:38.304790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:38.305821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:38.305821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.305821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:38.313559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 374/733] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:38.314999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.357735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:38.378027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.381137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.442658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11b24d-0b6d-4b1d-a24d-5d82d3bd1df9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.445048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:38.488836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.490579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.491248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.491734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.505040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.507596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:38.524710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:38.525762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:38.526709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:38.526709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.526709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:38.528901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 375/734] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:38.530237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.612443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 360/735] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:36:38.614498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.659529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49b243-bbba-48dd-b94d-a127f0dfa750 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.700683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.701502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.701827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.702111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.702892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:38.703273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.704731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.718809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 376/736] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:38.719707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.743632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:38.747142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:38.759760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a7e65-3907-4998-b2b1-c96db0cfe3b8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.807943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.808982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.809475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.810006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.810753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:38.811059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.812489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f140285-3934-4a3f-a6e3-be32f9f75df4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 361/737] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9797d-3068-41ed-b62e-c0865c050e7c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9797d-3068-41ed-b62e-c0865c050e7c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9797d-3068-41ed-b62e-c0865c050e7c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 362/738] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:38.830662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9797d-3068-41ed-b62e-c0865c050e7c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.860864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfc642-4158-463a-be20-4ba487a544c0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:38.906735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:38.907663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:38.908116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:38.908387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:38.920422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:38.921936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:38.940286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:38.941452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:38.942503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:38.942503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:38.942503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:38.945339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 377/739] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:38.946268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:38.967367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9797d-3068-41ed-b62e-c0865c050e7c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.011945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.012722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.013070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.013492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.023855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.025172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.026527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f976ef51-3cc9-46a1-a6b3-3bf5dd6a1b8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:39.042208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:39.043392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:39.044353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 363/740] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:39.044983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.045625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.045625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.065805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:39.068206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:39.077878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.084576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72823f75-9da2-4beb-a1cf-9812da3ac124 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.127387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.128050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.128321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.128594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.139970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.141895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.169680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 364/741] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:39.169680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:39.170137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:39.171675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 378/742] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:39.173223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:39.374333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ef3d-506e-4784-9cbd-d53019b5e90e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.407416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f85f-1d55-493e-bc24-f09e7453e9e3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.419618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.433310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.433882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.434164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.434485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.435811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:39.436114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.438014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.465899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 379/743] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:39.468373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:39.486777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.487996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.488555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.489261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.491799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:39.492423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.495661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.525705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 365/744] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:39.525705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:39.528361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.617866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c4ea-2259-4c2c-ba73-94a3652c2c69 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.686243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.687243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.687780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.688280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.689596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:39.690175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.693513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.695357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd1bd-11f0-44a8-b5cf-c8c78e011569 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.717927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 380/745] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:39.719734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:39.763073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.764185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.764775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.765204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.810391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.903584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.906542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.910220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:39.918340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1a508-85e5-40fe-beb6-2ed30268daad None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:39.967662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:39.970082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:39.970587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:39.971073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:39.972169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:39.974808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:39.987096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:39.988270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:39.989230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 381/746] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:39.990923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.073876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 366/747] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:36:40.074796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.125968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd4479-69de-4f43-a669-d1baede3a97b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.143643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:40.180512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.181626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.182165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.182629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.195679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.198395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:40.222626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:40.225676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:40.227533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:40.233936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 382/748] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:40.235320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.281172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934c762-b6d0-431d-9306-891bcf1aeef5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.284922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:40.331802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.332431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.332727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.332997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.334235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:40.334430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.335791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.345736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 367/749] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:40.346697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.427832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61392a-0abd-4620-90fe-a3aa7f0de214 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.476602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.477179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.477460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.477763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.478485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:40.478791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.480240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.489699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 383/750] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:40.489699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.539330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f836396-ca81-4cbe-9e09-9495e9e95d91 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.584595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:40.597986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.600696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.601610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.602604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.625042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.627898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:40.662310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:40.664144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:40.666643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:40.666643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:40.666643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:40.669763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 368/751] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:40.671735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.713204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:40.742794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9819d-8208-4410-aaba-c40ec1383d7f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.809044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.810581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.811359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.812109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.813635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:40.814310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.816856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.846053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 384/752] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:40.847440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:40.888600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e3cd0-647a-4547-af55-b8d23707f5f7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:40.947184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:40.947805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.948068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.948400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.948983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:40.949274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:40.950785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0facd-10e3-4355-a73b-e1de66689ae9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:40.964498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 369/753] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:40.964498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6c1f7-27a9-42b0-a191-297a615363a9 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:40.964498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6c1f7-27a9-42b0-a191-297a615363a9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:40.964498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6c1f7-27a9-42b0-a191-297a615363a9 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:40.964498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 370/754] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:40.965224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6c1f7-27a9-42b0-a191-297a615363a9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:41.030291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:41.048146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601356-9749-4dcc-a49d-2ebd3c1b35f9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:41.096728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:41.106067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:41.106805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:41.107078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:41.107362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:41.108014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:41.108443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:41.109864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5b4b-b61c-4286-b2dc-b01d6aeeac71 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:41.128339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 385/755] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:41.130113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:41.131264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:41.131264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:41.151726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6c1f7-27a9-42b0-a191-297a615363a9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:41.155962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:41.157592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:41.213979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:41.214928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:41.215233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:41.215512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:41.216475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=2ab3501f173341538094ffeb2a59f004) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:41.216943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:41.219598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:41.241726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 371/756] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:40 2022] GET /identity/v3/role_assignments?user.id=2ab3501f173341538094ffeb2a59f004&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:41.241726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:41.308449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 386/757] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:41.310490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:41.572607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:41.581968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6734ed-d22c-4a59-b1cd-26455c0687c5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:41.630032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d1aa-26f9-4c27-8a22-61e5c6f6aa37 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:41.693684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:41.693684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:41.693684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:41.693684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:41.694321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] RBAC: Adding query filter params (interface=public, service_id=afe739ea258c43f8b4adccbf6e4a673f, region_id=RegionOne) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:41.694321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:41.694540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:41.696710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:41.697382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:41.697699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:41.697972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:41.711251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:41.712875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:41.716878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 372/758] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:36:41 2022] GET /identity/v3/endpoints?service_id=afe739ea258c43f8b4adccbf6e4a673f&interface=public®ion_id=RegionOne => generated 194 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:41.726512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:41.743199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:41.744906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:41.746779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 387/759] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:41.752672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:41.754090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:41.954515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb15cb7-5205-4988-8e0d-5dfe937615d9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:41.999531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ca6cd-f57b-4e3b-a0e4-13603495ee98 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:42.015597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:42.016330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.016684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.017033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.018996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:42.019498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:42.022363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:42.052947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 388/760] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:42.054356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:42.058973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:42.059982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.060483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.060964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.061223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:42.062799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:42.065608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:42.077867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 373/761] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:36:41 2022] GET /identity/v3/services => generated 772 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:42.077867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:42.296256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:42.339981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6cff8-07d1-4dbb-bfab-6d77ca346b13 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:42.358739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d38f7-8945-4564-88b8-9cf9a45789de None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:42.409636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:42.410542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.411921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.411921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.411921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:42.415565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:42.421316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:42.422163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.422629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.423067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.424014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=f7848ff43a794bd4aa35bdff59a53c73) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:42.424428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:42.426239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:42.429547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:42.432417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:42.432417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f2310-2d32-43a6-bac6-e62e3d6c753a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:42.432870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 389/762] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:42 2022] GET /identity/v3/services/network => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:42.434452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:42.444324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 374/763] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:42 2022] GET /identity/v3/role_assignments?user.id=f7848ff43a794bd4aa35bdff59a53c73&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:42.549062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4a31-575e-4495-89ac-6ccf84b849d5 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.549542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4a31-575e-4495-89ac-6ccf84b849d5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.549998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4a31-575e-4495-89ac-6ccf84b849d5 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.553963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 375/764] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:42.560502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.561192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.561732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.583114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:42.584855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:42.586146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:42.677602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251f69-afca-4635-9e27-f4a1be35d5b0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:42.747888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:42.748891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:42.753977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 376/765] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:36:42.756068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:42.756763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:42.757837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] RBAC: Adding query filter params (name=network) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:42.758375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:42.760278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:42.771372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 390/766] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:42.777873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:42.935247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:42.981536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45529-2c58-47de-9b36-6b81db86215a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:43.033416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:43.034418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:43.034897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:43.035380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:43.037154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] RBAC: Adding query filter params (type=network) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:43.037784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:43.039935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:43.050690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 377/767] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:43.057135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:43.131894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:43.182664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180616-6355-4ccb-aaaa-b79b5e81d722 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:43.220184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:43.220741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:43.221025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:43.221302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:43.222062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] RBAC: Adding query filter params (interface=public, service_id=24a4654ab3284bb18a6368bfe0802725, region_id=RegionOne) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:43.222379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:43.225868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:43.230114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:43.245411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 391/768] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:36:43 2022] GET /identity/v3/endpoints?service_id=24a4654ab3284bb18a6368bfe0802725&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:43.258840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:43.407231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784aeab-8185-4bf9-8f2f-14586cd54f26 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:43.518983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:43.543712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:43.544499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:43.544810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:43.545081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:43.546852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc98025-1f68-41cf-b543-d3ac584b85fe None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:43.591495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:43.608212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:43.610174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:43.611895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 378/769] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:43.622715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:43.643511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:43.644574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:43.645054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:43.645524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:43.648186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:43.650937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:43.662874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 392/770] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:36:43 2022] GET /identity/v3/services => generated 772 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:43.813986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1c8c-a780-498f-ba38-76e7297511a3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:43.903677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:43.904732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:43.905222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:43.905744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:43.906999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:43.907537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:43.910246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:43.929204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 379/771] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:43.938593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:44.043871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:44.132000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c3c7-da05-4308-8da1-b8ed6d4d3ca5 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:44.215206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:44.216378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:44.216879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:44.217362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:44.244057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:44.245558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:44.290417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:44.292343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:44.294141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:44.294141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:44.294141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:44.294141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 393/772] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:44.306783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:44.429585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:44.511477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9592ed-c851-479e-9a8c-6f74840af1bd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:44.579437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:44.580444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:44.580929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:44.581422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:44.582587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:44.583127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:44.585776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7378f5-3970-4fb4-8d68-a6744db86b9f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:44.609156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 380/773] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:44.618106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:44.767464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac75d8-b7bf-4f4f-9849-bb2c59b243f1 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:44.767917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac75d8-b7bf-4f4f-9849-bb2c59b243f1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:44.768334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac75d8-b7bf-4f4f-9849-bb2c59b243f1 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:44.774949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 381/774] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:36:44.780555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:44.781056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:44.781556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:44.833959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:44.835226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:44.853865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec461f-f560-4e19-a170-1c30a176d061 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:44.890055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:44.993802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:44.994867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:44.995354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:44.995828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.016108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 382/775] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:36:45.017223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:45.020071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:45.059253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:45.060304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:45.061273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 394/776] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:45.067174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:45.134721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:45.228900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20508c-2ee8-49b8-b9e3-4eba8cfd172f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:45.347373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:45.348320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.349572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.350783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.351748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:45.352280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:45.355109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8980b-7686-4f62-9d3c-ad31213ddc3a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:45.393958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 383/777] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:45.401522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:45.587514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082235c-6fbe-4b9a-97c2-0fa7e38d1bc4 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.587991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082235c-6fbe-4b9a-97c2-0fa7e38d1bc4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.588756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082235c-6fbe-4b9a-97c2-0fa7e38d1bc4 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.592959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 384/778] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:36:45.598461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf5cb-453a-45e4-96cf-c62dc995b572 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:45.600491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.600967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.601384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.632874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:45.634957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:45.642192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:45.643235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.643741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.644214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.657845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:45.659702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:45.679282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:45.680774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:45.682227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:45.682227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:45.682227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:45.684718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 395/779] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:45.685964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:45.717911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:45.782254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 385/780] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:36:45.783601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:45.846445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6175-bf2d-4449-a322-d7f9877fc0eb None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:45.866217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:45.893586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:45.896292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.897257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.898044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.921574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-de193066-cfc4-4e99-b12d-98140652046e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:45.929872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:45.931103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:45.931943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:45.935874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 396/781] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:45.938493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e02f5-f5b1-40b2-b755-260880389be5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:45.939306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e02f5-f5b1-40b2-b755-260880389be5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:45.940032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e02f5-f5b1-40b2-b755-260880389be5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:45.943416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 397/782] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:45.945385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e02f5-f5b1-40b2-b755-260880389be5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:45.993007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94ac7b-0954-4cdd-b7bd-a140bd4147cc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.038290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.039373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.039922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.040402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.041330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:46.042014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.043712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee024d3-b362-46f0-9ff3-f97f7d076d96 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.058695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 386/783] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:46.060471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.060983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.061459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.074112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.084852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:46.086816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:46.102774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e02f5-f5b1-40b2-b755-260880389be5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.173467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.174129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.174423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.174686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.175390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:46.175723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.178257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.189727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 398/784] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:46.189727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.194509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 387/785] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:46.195449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.223881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.360516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320331-b7f6-49eb-8382-7bd655eb1a0b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.393053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4a063-1d17-409e-9a6d-8fc0a46aaac0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.412379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.413190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.413572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.414030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.430890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:46.437680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:46.439478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:46.441393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:46.445882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 388/786] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:46.448511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.457033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.457945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.458267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.459110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.468749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.470321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:46.483648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:46.484655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:46.485570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 399/787] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:46.486684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.491909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.583179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.610212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c230bf-5019-440a-8590-3762f209b777 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.643653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73c57e-dda6-4371-bbd1-bc6de43f3124 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.658569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.659212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.659937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.660238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.678697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.681395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.701039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.702083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.702540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.702992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.704170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:46.704674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.707249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:46.709318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:46.710308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:46.711210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:46.711210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:46.711210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:46.711210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 389/788] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:46.712761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.731473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 400/789] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:46.731473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.825713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.872178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c35f5-4a31-4e7b-8cc5-1b947a8713c6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.873045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa1b1b-aa5c-46e5-9e52-c06d13632160 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:46.912369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.913093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.913278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.913545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.914262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.915682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.924147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:46.924761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:46.925038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:46.925308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:46.925594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:46.928402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 11:36:46.931220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:46.933217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:46.933217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:46.933600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 401/790] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:46.933600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:46.933909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:46.935847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:46.948325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 390/791] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:46.949599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.069607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6373a-4e8a-4de2-955a-3a364d82ac4b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.075896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555f1c-7b04-4e3e-9238-760206ab99f7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.106246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:47.114935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.115898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.116349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.116751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.117868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:47.118385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.120839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6aab0-cfc6-4e9a-86fc-ea52ae0b868b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.123829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.124832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.125205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.125607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.138290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 402/792] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:47.140520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743cf-e7bb-4e37-8dbd-da17f66c5824 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.141093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.141241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743cf-e7bb-4e37-8dbd-da17f66c5824 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.141668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743cf-e7bb-4e37-8dbd-da17f66c5824 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.142709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.146295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 403/793] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:47.146523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743cf-e7bb-4e37-8dbd-da17f66c5824 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:47.165408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:47.166538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:47.167468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:47.167468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:47.167468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:47.167468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 391/794] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:47.167468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.225103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:47.273739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743cf-e7bb-4e37-8dbd-da17f66c5824 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.291272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f9a27-215a-4e56-8a74-36d24f666ca4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.316613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.317536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.318474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.319073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.320164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:47.320683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.323106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.330799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 404/795] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:47.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.342817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.343497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.343778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.344058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.346151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=9260fd9f9c524d20a9025595a53ade4e, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.347618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28710a-3ace-4535-9e9c-e280c37777bd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.362223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 392/796] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:47 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:47.363476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.363758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.364083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.384410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:47.386018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:47.419609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:47.501806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 393/797] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:47.504710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.512090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e4bd-cbce-4796-a856-a8b06a47da8f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.579292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:47.592461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.594124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.594900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.595666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.617115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.620133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:47.648498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:47.650726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 405/798] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:47.652779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.747187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddfc00-63ab-4d5c-8b84-6bd9c869d76c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.822997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.824580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.825187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.827040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.828257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:47.828769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.830693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.846162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 394/799] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:47.847540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:47.876966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522432b3-3ed8-40ca-b140-a13e17add284 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:47.898120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:47.924130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:47.924831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:47.925176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:47.925444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:47.926139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:47.926474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:47.929759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:47.940302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 406/800] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:47.940934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.003697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:48.012173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbf2d-3fca-4857-9b97-73783ecec044 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.054899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.055537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.056374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.056702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.057472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:48.057822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.059742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.087665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 395/801] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:48.087665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.096841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7298-afde-4d49-be6a-991d657d9094 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.144062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.144856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.145063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.145326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] PATH_INFO: `/v3/services/network` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.146630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.148631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:48.165675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:48.167697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:48.169366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 407/802] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:47 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:48.171012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.263381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:48.279815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23661280-355f-4785-a3df-41266a701fd2 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.340727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.341742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.342228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.342696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.354818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.356302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.365007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c9017-5d0c-4c86-a712-1b6dfa02e44b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:48.369945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:48.371769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 396/803] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:48.373585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.393211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:48.443517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.444492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.444969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.445419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.446721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.449380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.485686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 408/804] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:36:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:48.485686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.534923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bcd7b-34f6-488c-a34d-3600d58ad775 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.612831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.613894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.614571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.615171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.636556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.639333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:48.668306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:48.669628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:48.670560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:48.670560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:48.670560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:48.673957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 397/805] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:48.673957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.792020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:48.842137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:48.842372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb636b9e-b9cb-49e9-9d1f-5fd01c755c8b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.851428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e3a-95ab-4933-bb1a-74a823f40888 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:48.884476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.885426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.885835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.886202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.887084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:48.887495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.889233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.897915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:48.898808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:48.899230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:48.899637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:48.900514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] RBAC: Adding query filter params (name=network) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:48.900959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:48.902588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:48.908503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 409/806] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:48.909619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:48.916505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 398/807] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:48.917846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.034286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c1b1-52b2-4409-a26d-dcc0f7bd82e8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.042519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf32d5-b032-42fc-9918-bc256ed9414d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.069721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.077302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.078377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.078876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.079348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.080222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:49.080769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.082448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.089290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.089952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.090227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.090494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.091144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] RBAC: Adding query filter params (type=network) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:49.091446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.092871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.099370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 399/808] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:49.100412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.103185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 410/809] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:49.111536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.143236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.236024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b5a4-ee7c-4c82-b129-8437b2ccce2c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.273260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db578ab-a7d2-412f-a20a-e4bc2d00214f None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.285244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.285946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.286223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.287162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.295223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.296828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:49.309497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:49.310623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:49.311530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 411/810] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:49.312009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.344567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.345760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.346256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.346746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.368181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.371201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:49.402673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:49.403649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:49.404534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:49.404534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:49.404534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:49.409606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 400/811] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:49.410926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.412060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.431826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774c32-4afe-42e2-b185-cfffb57a8a5f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.473314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.474517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.474874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.475348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.476179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.477593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.488614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.513087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 412/812] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:36:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:49.515009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.622663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a560b-9103-4738-801c-43ff6aa6d45c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.741235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.741924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.742203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.742473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.743218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:49.743528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.750518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.768912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 401/813] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:49.777934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.779789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de517-9f6d-49c5-8d72-fb883bfdf0df None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:49.825302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:49.826055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:49.826333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:49.826604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:49.827317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:49.828029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:49.829464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:49.846073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.851672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 413/814] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:49.859793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:49.950335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:49.963840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7cdc-54e9-4f18-987c-7a0a715931a8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:50.016049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:50.016601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.016892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.017139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/2ab3501f173341538094ffeb2a59f004/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.034998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=2ab3501f173341538094ffeb2a59f004, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:50.037026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:50.058345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 402/815] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:49 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/2ab3501f173341538094ffeb2a59f004/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:36:50.059862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2f30-b3ed-4585-be32-8e4681011b25 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:50.133979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:50.135744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.136244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.136716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.152029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:50.155048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-3c3ed012-ed33-449c-92a2-53af3323ac0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:50.178253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:50.179242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:50.180137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:50.181057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 414/816] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:50.190086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:50.287375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0d2a-4be4-4136-b031-5a97595f65fd None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.287765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0d2a-4be4-4136-b031-5a97595f65fd None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.288238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0d2a-4be4-4136-b031-5a97595f65fd None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.293812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 415/817] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:36:50.296191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.296468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.296737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.319885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:50.321200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:50.391367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:50.421804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 416/818] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:50.471133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6404-f85e-4f4c-8b49-546e9ff5e757 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:50.512980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:50.513609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.513918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.514183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.514902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:50.515205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:50.516720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:50.532084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 403/819] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:50.542925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:50.682604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d3e8f-2432-4b03-92e8-aa1f4c848371 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:50.709770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:50.713233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:50.729852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:50.730908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.731192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.731457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/f7848ff43a794bd4aa35bdff59a53c73/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.734550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=f7848ff43a794bd4aa35bdff59a53c73, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:50.736317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:50.758286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 417/820] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:36:50 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/f7848ff43a794bd4aa35bdff59a53c73/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:36:50.865386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7838-f333-4ebf-92cb-a415c5714bee None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:50.913242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:50.914042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:50.914324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:50.914596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:50.929880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-0144edcc-99aa-44d4-9da5-0699875ccbfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:50.935023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:50.936049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:50.936990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:50.941561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 404/821] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:50.945343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.041855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:51.072110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c00043-4d6e-4919-8f07-6e59d7f56dc0 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.073078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c00043-4d6e-4919-8f07-6e59d7f56dc0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.073559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c00043-4d6e-4919-8f07-6e59d7f56dc0 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.075530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 405/822] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:36:51.081800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.082471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.082957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.110589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:51.113827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:51.136101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fd713-3ac0-4dcf-a755-0f91ea255a66 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:51.207815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:51.210427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.210940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.211424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.212633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:51.213170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:51.215537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:51.217683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:51.233820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 406/823] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:36:51.241828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 418/824] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:51.248426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.427822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306ea21-1e6f-49d8-be7a-5434e0ed45ce None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:51.466390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:51.467038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.467310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.467580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.479781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:51.481283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:51.497927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:51.499181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:51.500276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:51.500276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:51.500276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:51.502388 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 407/825] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:51.514815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.531215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.604674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:51.691087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9894e-c458-4ece-914c-b8d542097b68 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:51.737530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b58a49-d3cc-4dc7-98d5-5872d999029d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:51.739328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:51.740254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.741278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.741782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.742586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:51.745670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:51.757920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:51.759202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:36:51.760482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:51.769252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 408/826] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:36:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:51.770032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:51.771097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.809573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:51.811049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:51.811557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:51.812058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:51.812960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:51.813493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:51.815216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:51.834021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 419/827] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:51.843205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:51.981681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e85bc-632b-41d7-b6b5-d1fdb175a076 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:51.990447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7807c-7a19-43aa-b7d3-a2afed8423d0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:52.001692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:52.059686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:52.061426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.062412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.063163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.064654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:52.065603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:52.067902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:52.069019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.069453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:52.069611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.069978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.080544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 409/828] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:36:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:52.091135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:52.092101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:52.094810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:52.122733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:52.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:36:52.127000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:52.136311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 420/829] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:36:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:52.142835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:52.211595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:52.339843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde78cb6-94ca-460e-99e6-b3c970551499 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:52.382537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95587551-239f-48f0-96b3-983b95dfe10c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:52.385531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:52.386139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.386467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.386709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.387337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:52.387644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:52.389145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b20ae-c233-4b60-9674-bf446d39ff21 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:52.399603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 410/830] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:36:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:52.399603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c559-bdd0-4f98-81ec-29284fd2b8e1 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.399603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c559-bdd0-4f98-81ec-29284fd2b8e1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.399603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c559-bdd0-4f98-81ec-29284fd2b8e1 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.402838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 411/831] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:52.404342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.404658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.404906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.430723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:52.433091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:52.447463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:52.448418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.449550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.450038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.451012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:52.451476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:52.453392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:52.479722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 421/832] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:36:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:52.491705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:52.500065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:52.604085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 412/833] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:52.695593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:52.715124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba160f-0b22-4b6a-8636-53b573143add None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:52.786816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:52.787871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:52.788355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:52.788833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:52.889394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:52.922489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:52.924317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.047254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:53.093741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 422/834] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:36:53.100027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.120004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68271a3-5eaf-4e4e-ab26-18402605c68d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.176467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.178244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.179083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.179891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.210338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:53.223543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:53.226130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:53.227764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 413/835] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:53.234688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.294410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ec6a-2c5a-4c34-b9d5-49fe2e16064d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.309517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:53.368790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.370128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.370739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.371337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.388326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.391166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:53.414556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:53.415687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:53.416603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:53.416603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:53.416603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:53.424088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 423/836] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:53.425929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.478502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0fd7e-853f-46fc-b698-4373b355906a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.483000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:53.534574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.535556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.535731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.536114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.537265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:53.537737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.539345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.554162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 414/837] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:53.556505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.570771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70792-b2eb-4756-b681-09318ecfc736 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.621135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.622149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.622794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.623373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.624215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:53.624985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.627508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.652030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 424/838] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:53.655385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.674107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:53.693503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867c76c-041f-4717-a138-216be56ded39 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.742052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.743982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.744512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.744997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.761515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.763428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.775414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:53.780646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09283307-438e-4d12-ad0c-3db8dc1b0bf3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:53.785283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:53.786832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:53.787735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:53.787735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:53.787735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:53.790016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 415/839] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:53.795553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.836804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.838183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.839521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.840688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.852696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.855454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:53.871380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:53.872348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:53.873341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:53.881472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 425/840] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:53.883423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:53.945566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534dbea-092a-46a1-a877-4df57488a348 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:53.985403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:53.986483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:53.986971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:53.987448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:53.988327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:53.988879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:53.990670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ac192-f83f-45cb-a5e5-be6835562af2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.001530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.006638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 416/841] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:54.008593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a587e8-c9f4-4eed-8f5b-c9b097880ae7 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.008878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a587e8-c9f4-4eed-8f5b-c9b097880ae7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.009152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a587e8-c9f4-4eed-8f5b-c9b097880ae7 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.011061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 417/842] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:54.011774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a587e8-c9f4-4eed-8f5b-c9b097880ae7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.037668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578318a-d68d-489d-b25c-f247f7361bf8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.092147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.093318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.093830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.094313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.095198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:54.095708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.097960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4f8bc-450c-4449-8289-74c73a0661e4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.113962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 426/843] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:54.115982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.116300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.116537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.138576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:54.139808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:54.154682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a587e8-c9f4-4eed-8f5b-c9b097880ae7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.157756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.204032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.205399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.205902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.206366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.220281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.229925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 427/844] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:54.230178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.230322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:54.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:54.245431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:36:54.247306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:54.257845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 418/845] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:54.262904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.358762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.421272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f926a90-0be2-4d9f-8386-b80aede045d4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.462488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b74a5-63e0-4b72-9b62-4218db4503ad None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.494054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.495627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.496415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.497192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.499047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:54.499931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.503129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.516510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 428/846] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:54.518624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.534842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.535903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.536933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.538118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.539706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:54.540272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.543367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.559387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.576501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 419/847] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:54.586223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.699036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667e90-17ab-4bfd-8cbe-6bf578892998 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.748857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.749614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.750002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.750342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.773029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:36:54.785297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:54.787797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:54.790212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 429/848] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:54.792713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1da4a-83f8-4c92-89b7-5d45934ba548 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.795953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:54.817334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.839889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.840522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.840794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.841068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.913702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:54.948199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239cac-8583-4f22-8e2e-a3ae8aef81f9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:54.956580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.959408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:54.993131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:54.993784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:54.994074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:54.994339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:54.995195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:54.995501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:54.997078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.013729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 430/849] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:55.014999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.097217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 420/850] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:36:55.107118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.148166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4127-31db-4331-aa36-bda38670dc14 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:55.166075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:55.198221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:55.198876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.199156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.199417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.203687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:55.215267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:55.217585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0d21b80e-e9ac-4934-98d3-cc5b9399d04d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:55.252177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:55.253473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:55.254943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:55.254943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:55.254943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:55.256323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 431/851] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:55.261664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb3e9-8a9f-499d-a18c-f0df8d84e0a5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.261664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb3e9-8a9f-499d-a18c-f0df8d84e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.261664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafb3e9-8a9f-499d-a18c-f0df8d84e0a5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.263015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 432/852] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:55.264432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafb3e9-8a9f-499d-a18c-f0df8d84e0a5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.327272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f7468-a113-4b34-90d2-786f44a6a537 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:55.369270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32378509-15da-4511-a61a-a87577d14244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:55.369857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32378509-15da-4511-a61a-a87577d14244 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.370131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32378509-15da-4511-a61a-a87577d14244 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.370396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32378509-15da-4511-a61a-a87577d14244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.382713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32378509-15da-4511-a61a-a87577d14244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:55.384210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32378509-15da-4511-a61a-a87577d14244 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-32378509-15da-4511-a61a-a87577d14244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:55.401503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:55.403555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 421/853] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.405433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.430315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:55.432157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:55.432620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafb3e9-8a9f-499d-a18c-f0df8d84e0a5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:55.484598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:55.485810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.486376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.486943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.488136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:55.488795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:55.491729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.512094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 433/854] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:55.513273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.532781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:55.544107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 422/855] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:36:55.545431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.603190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:55.706000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b2f9-5c18-4e92-b0ea-83a3bdfb7127 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:55.732292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e88438-cdab-413c-bc1c-3cf6bd47de19 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:55.800865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:55.802534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:55.803906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.804829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:55.805323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.806151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.806517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:55.807762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:55.808678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:55.810538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:55.812592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.828127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:55.830104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:55.843225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 434/856] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:55.857676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:36:55.872832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:55.873897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 11:36:55.874842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:55.878418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 423/857] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:55.894985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.019901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:56.072578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaba4b1-fcef-4878-bb0c-852ad71c2c9e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:56.078592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718dbe0-06e5-47ba-9242-3c44828d4e29 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:56.092157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:56.117985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:56.118614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:56.118887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:56.119155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:56.119916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:56.120219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:56.121714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:56.136830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:56.137906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:56.138327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:56.138630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:56.143635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 424/858] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:36:56.144527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.159317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:56.160815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:56.183723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:56.184713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:56.185594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:56.186647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 435/859] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:36:56.187270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.297140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d954c5-48d2-4a3c-a34a-9af7dae29b25 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:56.326179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:56.361820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:56.362852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:56.363335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:56.363865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:56.365608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:56.368541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:56.397046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 425/860] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:36:56 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:36:56.398641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.429986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea4c3d-2e8a-4896-babe-a7a74abeff42 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:56.516607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:56.519155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:56.520001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:56.521402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:56.585893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:56.654520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ea1b-4d1f-4454-9b4c-54564f7425f6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:56.655814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:56.657428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:56.713181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:56.714179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:56.714460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:56.714742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:56.715583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:56.715992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:56.718832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:56.748103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 426/861] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:56.758778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.818904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:56.916380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 436/862] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:36:56 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:36:56.922057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:56.950790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cc452-efe0-4fd6-a93b-6502a370401b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:57.010094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:57.046981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:57.048963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.050012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.051320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.053041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=2ab3501f173341538094ffeb2a59f004) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:57.053909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:57.058961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:57.069727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7ae8-a4dc-481c-a179-4406a691f360 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:57.086784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 427/863] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:56 2022] GET /identity/v3/role_assignments?user.id=2ab3501f173341538094ffeb2a59f004&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:57.133305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:57.134047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.134327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.134604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.153910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:57.156439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-087cb7f7-7fee-46d8-9bb6-ac8bb5680e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:36:57.190124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:57.191912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:36:57.193730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:36:57.193730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:36:57.193730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:36:57.197537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 437/864] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:57.206431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:57.439375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323449b-445d-4f9f-b30e-d186898e7ee8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:57.442244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:57.520300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782b7-d35d-4185-acd9-9cecb9c70c0c None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.521719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782b7-d35d-4185-acd9-9cecb9c70c0c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.521719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3782b7-d35d-4185-acd9-9cecb9c70c0c None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.524549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 438/865] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:57.525074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:57.526225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.526832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.527291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.528382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:57.528890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:57.531987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:57.534621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.535094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.535550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.553376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 428/866] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:57.562816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:57.570700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:57.572932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:57.680348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d46d-a6de-4197-ad1d-98d23c651e26 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:57.717556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 439/867] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:36:57.720956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:57.721627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:57.721920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:57.722171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:57.730443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:57.731938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:36:57.745500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:57.747671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:57.749490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 429/868] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:57.761418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:57.852689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:57.954157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2856-3018-4433-b7e5-12feddf4aebf None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:58.015481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:58.025962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:58.027509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:58.028281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:58.029045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:58.030761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:58.031616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:58.034789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:58.065835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 440/869] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:58.067656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:58.073048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:58.225156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7e44-0fb8-4212-912e-f7c610a9684a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:58.282279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:58.283311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:58.283827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:58.284327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:58.285255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256971ae-c9a2-44ef-a73a-fd22729ae0b6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:58.285426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:58.287315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:36:58.299229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:36:58.300368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:36:58.301334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:36:58.302758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:58.304919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 430/870] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 6 11:36:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:36:58.311556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:58.349883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:58.350961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:58.351396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:58.351820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:58.352981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=f7848ff43a794bd4aa35bdff59a53c73) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:58.353477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:58.355324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:58.363238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 441/871] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:36:58 2022] GET /identity/v3/role_assignments?user.id=f7848ff43a794bd4aa35bdff59a53c73&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:58.468254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:58.504829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ecc7-c4a9-4b1d-a236-719f4fa716b8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:58.581758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:58.582731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:58.583832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:58.584313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:58.585504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:58.586077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:58.588692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f5051-05f9-45d9-924a-56b22372d6b5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:58.599273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 431/872] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:36:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:58.605087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:36:58.819729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f35c9-f7d3-49dc-8456-afe12d44093f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:36:58.888407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:36:58.889517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:58.890028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:58.894909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:58.894909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:36:58.894909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:36:58.899016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ba9f7-517f-4968-b17a-ae26b94022e1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:36:58.905303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 442/873] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:36:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:36:59.812196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74e2-ce9e-4bf6-9707-a2f07d32f4cb None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:59.812819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74e2-ce9e-4bf6-9707-a2f07d32f4cb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:59.812819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74e2-ce9e-4bf6-9707-a2f07d32f4cb None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:59.814420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 432/874] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:59.820966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:59.821527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:59.821527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:59.844484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:59.845758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:59.868037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ae460-1892-4d62-ba62-375986b29399 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:59.868539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ae460-1892-4d62-ba62-375986b29399 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:59.870761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ae460-1892-4d62-ba62-375986b29399 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:59.875500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 433/875] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:36:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:36:59.883420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:36:59.883887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:36:59.884344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:36:59.885536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:36:59.918608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:36:59.921029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:36:59.957900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 443/876] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:00.031877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 434/877] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:36:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:00.347142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.372882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.526618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:00.530076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7e51a-10b8-412f-8f69-c73af381bc6e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:00.568631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf7030-a5b1-4c60-b1af-a4635fca5375 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:00.591104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:00.592241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:00.592829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:00.593448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:00.594776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:00.597114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:00.616606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:00.617962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:00.619045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:00.620225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:00.621120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:00.621560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:00.622004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:00.628811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 444/878] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:00.634499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.636288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:00.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:00.644033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:00.644907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 435/879] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:00.655004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.681918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:00.771059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1cf49-4229-4091-8d30-2131c5b50496 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:00.799255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f53b0-542a-4020-959f-891f41392c2e None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:00.813156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:00.814087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:00.814471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:00.814848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:00.815629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:00.816049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:00.818775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:00.829168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 445/880] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:00.830036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.866501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:00.868868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:00.869714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:00.870630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:00.872335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:00.873243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:00.876426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:00.896085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 436/881] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:00.904063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:00.908629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:00.976713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:00.988280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0b96b-8d1f-4aa9-a755-e594196e3bdf None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.047390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.048420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.049047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.049607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.050817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:01.051032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.053333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5771c6b-7f36-47fc-8e5c-7783b9a3b5da None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.059834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 446/882] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:01.062083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6202-729d-46f6-8db8-08a257cb1210 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.062540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6202-729d-46f6-8db8-08a257cb1210 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.063011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6202-729d-46f6-8db8-08a257cb1210 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.065534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 447/883] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:01.071561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061eed5c-0055-4181-b277-2a0411d0f38d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.073758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6202-729d-46f6-8db8-08a257cb1210 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.136383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.137451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.137926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.138351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.156814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.159839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-21547bdf-0516-4511-906e-7b4483acd832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:01.187967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:01.189033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:01.189981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:01.189981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:01.189981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:01.190674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 437/884] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:01.192942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c8b3-1057-4b23-a349-3f460ef3f1dd None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.193438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c8b3-1057-4b23-a349-3f460ef3f1dd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.193971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c8b3-1057-4b23-a349-3f460ef3f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.226614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-16a5c8b3-1057-4b23-a349-3f460ef3f1dd None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:01.227742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6202-729d-46f6-8db8-08a257cb1210 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.228719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-16a5c8b3-1057-4b23-a349-3f460ef3f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:01.277749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.278903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.279402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.279883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.281194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] RBAC: Adding query filter params (interface=public, service_id=3783dfad31824d3281bab0f097077a4f, region_id=RegionOne) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:01.281769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.284355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.295538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 448/885] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/endpoints?service_id=3783dfad31824d3281bab0f097077a4f&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:01.296897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.303545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:01.323950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:01.324234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 438/886] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:01.325664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298e10-ce47-403e-bfa6-9e4a9eca3b44 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.325968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298e10-ce47-403e-bfa6-9e4a9eca3b44 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.326302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298e10-ce47-403e-bfa6-9e4a9eca3b44 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.327734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 439/887] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:01.328513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4298e10-ce47-403e-bfa6-9e4a9eca3b44 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.441379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa075778-b28c-452b-94e7-97212e268876 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.455807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4298e10-ce47-403e-bfa6-9e4a9eca3b44 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.502245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.502863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.503303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.503463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.504024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.506497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19d0eb-73e5-478a-b05b-dc0cac872f93 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.511921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.522382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.522694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 440/888] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/services => generated 1011 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:37:01.522849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.522849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.523115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.523115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.523400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:01.523556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.523699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.523699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.553297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:01.555742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:01.557343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 449/889] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:01.558896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.646010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:01.693298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 441/890] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:01.693298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.749126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:01.788524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d93e5-0398-4287-9196-f78585edd51e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.854712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.857321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.858135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.859050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.866017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374f97-8294-4f6d-8c4e-d36e2fe108d0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:01.876874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.880668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:01.908382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:01.909852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:01.913277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:01.917779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 450/891] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:01.920735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:01.948312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:01.949460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:01.950000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:01.950476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:01.951585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:01.958226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:01.971960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:01.975068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:01.978722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 442/892] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:01.982596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:02.071350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:02.140985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1454d8-3a48-45dd-89dc-96fab33a4e79 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:02.148122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69049e2-f68c-42a0-8e6e-43550a89b3bf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:02.201246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:02.202347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:02.202783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:02.203281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:02.203980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:02.204332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:02.204784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:02.205050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:02.205472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:02.206009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:02.206860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:02.207133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:02.207535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:02.209758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:02.225577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 443/893] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:02.225577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:02.228618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:02.245906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 451/894] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:37:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:02.247470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:02.414372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284810-cf27-43a4-b88b-31e0c1f94c6a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:02.425890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:02.463667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8893-7849-4ccb-a86c-1f2c317346a9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:02.466225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:02.467313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:02.467817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:02.468324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:02.469548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:02.472126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:02.499723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 444/895] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:02 2022] POST /identity/v3/services => generated 269 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:02.501441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:02.535908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:02.537912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:02.538395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:02.539026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:02.540244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:02.541162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:02.543670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:02.562797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 452/896] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:02.650487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb92a7-ebdf-4cfc-8396-1637d3e0350c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:02.705084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:02.706280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:02.706820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:02.707349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:02.756119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:02.801906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:02.804119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:02.942339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 445/897] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:37:02.952429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:03.040272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:03.091108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514a25-7db1-41ae-a91b-3c54fb512e1c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:03.134711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:03.135712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:03.135985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:03.136268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:03.149625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:03.151385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:03.170093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:03.172039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:03.173920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:03.173920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:03.173920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:03.173920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 453/898] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:03.184799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:03.265903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:03.310432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f7bc7-e033-43b7-9fba-ffbe03247fb8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:03.354057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:03.354690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:03.354970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:03.355235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:03.355912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:03.356209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:03.357706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:03.374231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 446/899] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:03.382996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:03.551994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f09837-e857-4d35-b02c-cc5181a53d47 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:03.564528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:03.593024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:03.593721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:03.593999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:03.594266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:03.602902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:03.604404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:03.617338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:03.618392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:03.619328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 454/900] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:03.626896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:03.810808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82fc6-6c0b-4650-9393-a89ddb349434 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:03.866313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:03.886135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:03.887783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:03.888533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:03.889276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:03.890934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:03.891803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:03.894922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a42b9-7498-45ce-8a92-0376c4575310 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:03.921438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 447/901] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:03.932227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:04.018325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7ea4e-3556-4a74-8a6e-a33ed3031786 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:04.019368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7ea4e-3556-4a74-8a6e-a33ed3031786 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:04.020151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7ea4e-3556-4a74-8a6e-a33ed3031786 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:04.024982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 448/902] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:04.032258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:04.033421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:04.034014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:04.067111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:04.069188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:04.164892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7ecf0-1916-4c75-b994-fa297c7a6a17 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:04.168734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 449/903] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:04.235457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:04.236257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:04.236680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:04.237144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:04.238453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] RBAC: Adding query filter params (role.id=86c64cac25b34439972a071e46a61bca, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=2ab3501f173341538094ffeb2a59f004) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:04.238929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:04.242945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:04.254574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 455/904] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:03 2022] GET /identity/v3/role_assignments?user.id=2ab3501f173341538094ffeb2a59f004&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=86c64cac25b34439972a071e46a61bca => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:04.471037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:04.602841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77a43-c1e9-4e03-8d4f-eaf21c263686 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:04.625612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:04.663831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:04.664949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:04.665450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:04.665990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:04.667135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:04.669910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:04.688441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:04.690828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:04.693454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 450/905] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:04.702154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:04.769269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:04.907132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046b6a-a1aa-4ec4-a75f-32898614bc11 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:04.948314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:04.949000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:04.949285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:04.949558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:04.950479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:04.950795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:04.952215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419e373-385b-48b4-b544-3157f48532d5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:04.958777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 456/906] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:04.964761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:05.039618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b311a-2c17-4750-8af8-27c4a9dbe667 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.039934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b311a-2c17-4750-8af8-27c4a9dbe667 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.040200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b311a-2c17-4750-8af8-27c4a9dbe667 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.041952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 457/907] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:05.047726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.048012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.048282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.072001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:05.073438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:05.164660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029bcb-a712-46c3-8375-040908d3d6f9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:05.181901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 458/908] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:05.232613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:05.233754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.234232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.234770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.235950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:05.236496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:05.239172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:05.249006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 451/909] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 6 11:37:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:05.256123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:05.308856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:05.456985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0299-ebb4-4768-bfc7-a8e8569f761f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:05.505194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:05.513076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:05.514003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.514452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.514938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.515818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:05.517622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f05f1-1f81-4f82-b2d9-53417842fd76 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:05.545133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 459/910] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:05.590340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:05.644654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dfcc-5117-4951-b741-4cad46e563c4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:05.695923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:05.696573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.696856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.697131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.697905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:05.699430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e809e-eab0-4b9e-898c-2dc46fe5b82c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:05.717582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 452/911] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:05 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:37:05.781936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fd09-d04f-487f-8b0a-2cf06da3b833 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.782480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fd09-d04f-487f-8b0a-2cf06da3b833 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.782808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85fd09-d04f-487f-8b0a-2cf06da3b833 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.784697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 460/912] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:05.795220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:05.796135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:05.796881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:05.828775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:05.830804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:05.962012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 453/913] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:06.497961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85fd09-d04f-487f-8b0a-2cf06da3b833 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:06.633276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:06.654250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85fd09-d04f-487f-8b0a-2cf06da3b833 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:06.694927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:06.695579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:06.695850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:06.696115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:06.696726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:06.698191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:06.708417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:06.710270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:06.712035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 461/914] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 6 11:37:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:06.720799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:06.762405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:06.861614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319aff0-a350-4e14-a562-fc299549de90 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:06.905996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:06.906533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:06.906816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:06.907057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:06.907715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:06.908018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:06.909449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:06.916133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 454/915] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:06.925396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:07.049492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4a428-4b16-4d5e-bfb4-dfbf3176d179 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:07.090452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:07.091079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.091430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.091685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.092345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:07.092642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:07.094146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d59c01-c0e5-40df-81a0-039e8d1046a7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:07.100823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 462/916] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:07.110954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:07.180061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666927d-aff5-49c1-88bb-4fce602f5fd9 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.181011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666927d-aff5-49c1-88bb-4fce602f5fd9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.181535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666927d-aff5-49c1-88bb-4fce602f5fd9 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.184570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 463/917] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:07.192999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.193626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.194403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.223405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:07.226691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:07.238222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:07.306952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:07.321514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a03bf-d302-47bb-bbd1-a3c86a7c9ed8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:07.351556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 464/918] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:07.376638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:07.377719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.378213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.378776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.380681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] RBAC: Adding query filter params (interface=public, service_id=07ce500033f94e86bde81d30b1de64c6, region_id=RegionOne) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:07.381197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:07.383789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:07.395070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 455/919] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:37:07 2022] GET /identity/v3/endpoints?service_id=07ce500033f94e86bde81d30b1de64c6&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:07.403241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:07.610413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b2e2c-efb1-4960-9f78-8a379d340f58 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:07.680992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:07.682063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.682553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.683044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.684131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:07.686835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ff41c-ea8d-4e7b-9842-b46ecbab944f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:07.699560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 465/920] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:07 2022] GET /identity/v3/services => generated 1497 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:37:07.879605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:07.953707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48c8a-c7f3-4b32-aa4f-c69acdf29319 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.953965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48c8a-c7f3-4b32-aa4f-c69acdf29319 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.954109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48c8a-c7f3-4b32-aa4f-c69acdf29319 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.955767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 466/921] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:07.961691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b628-5629-4dbb-8376-a5ef49f92fe5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:07.962053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b628-5629-4dbb-8376-a5ef49f92fe5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:07.962352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1b628-5629-4dbb-8376-a5ef49f92fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:07.983498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:07.992737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1bb1b628-5629-4dbb-8376-a5ef49f92fe5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:07.994848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1bb1b628-5629-4dbb-8376-a5ef49f92fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:08.044023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989880-9ffd-4bd7-b636-cd6ad6dcdcc4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:08.098576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:08.099781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.100104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.100431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.112986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-e58d1455-3582-43cb-9b31-5a35dcbcc91e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:08.118785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:08.119833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.120679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 456/922] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:08.121475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2493-6d34-4338-9a55-b967c32d959c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.121475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2493-6d34-4338-9a55-b967c32d959c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.122300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2493-6d34-4338-9a55-b967c32d959c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.127743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 457/923] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:08.128878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c2493-6d34-4338-9a55-b967c32d959c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.132364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 467/924] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:08.134865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.135120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.135845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.151622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:08.169902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:08.171865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:08.280175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c2493-6d34-4338-9a55-b967c32d959c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:08.318494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 468/925] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:08.332985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:08.334724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.335154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.335608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.336556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:08.337024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:08.338696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:08.349499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 458/926] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:08.356607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.463158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:08.470306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.531279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e005c-6c1c-4ae4-a8b1-fbe968a12313 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:08.598100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:08.599613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.600062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.600505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.624631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:08.627473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:08.633355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:08.655558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:08.656630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:08.657526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:08.657526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:08.657526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:08.657757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 469/927] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:08.658980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.669791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14484224-8d57-4ca1-8804-789e453a6c93 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:08.717748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:08.718338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.718639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.718912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.720194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:08.722027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:08.726343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING keystone.server.flask.common [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 11:37:08.752228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.sql.core [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 11:37:08.752228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 11:37:08.752228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [parameters: {'id': 'fd2d00b5c421438db78bdf966b6a98be', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 11:37:08.752228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87996) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd2d00b5c421438db78bdf966b6a98be.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd2d00b5c421438db78bdf966b6a98be. Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:37:08.774423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result.read() Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 11:37:08.775328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application yield resource Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise self.value Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application session.commit() Sep 06 11:37:08.776307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.session.flush() Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application compat.raise_( Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 11:37:08.777379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 11:37:08.780060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result.read() Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:37:08.782089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [parameters: {'id': 'fd2d00b5c421438db78bdf966b6a98be', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.785460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:08.788027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd2d00b5c421438db78bdf966b6a98be. Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 459/928] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 6 11:37:08 2022] POST /identity/v3/projects => generated 262 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:08.790174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.845468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea26a2-5ac6-42b7-8ca5-87a56b7c7004 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:08.869179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:08.896554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:08.897617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:08.898135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:08.898614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:08.899689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:08.902602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:08.920302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:08.921612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:08.922703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 470/929] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:08.924013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:08.962428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551dae-f9aa-46fb-a22a-bd6a419becaf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.018413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:09.027009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.028544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.029332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.029546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.030222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:09.031064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.033156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.059484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 460/930] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:09.059484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.065357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aa2b4-7710-4242-b7bd-1b01eaf0af0d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.111753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.112445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.112718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.112989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.122950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.124410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:09.145133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:09.146864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:09.147767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:09.154026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 471/931] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:09.154817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.183017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6b84-a18a-4978-945b-3dbbf3fef0c7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.227400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:09.233266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.233869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.234088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.234359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.235019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:09.235327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.236780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.248276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 461/932] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:09.248276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.310227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c960a-2d50-40b8-bc42-db5f59e4ed0b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.323827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:09.355363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.356062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.356304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.356563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.370643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.372883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.385921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afacfb-eff5-47fc-aa98-f3902b293df3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:09.394371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:09.396784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 472/933] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:09.407977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.446605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.447474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.447897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.448424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.449310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:09.449792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.452074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.484838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 462/934] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:09.484838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.633437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbadc80-6c2b-4717-b7da-9bad5f2a14b2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.637176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:09.706557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416fe8a-979a-4412-a406-4d2f79453a52 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.717150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.719061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.719538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.720002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.721879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:09.722410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.726402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.732255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:09.737840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 473/935] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:09.746330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.776852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:09.777820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:09.778267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:09.778686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:09.779605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:09.780084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:09.781970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:09.814453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 463/936] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:09.815071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:09.937551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ce9b-773c-405c-a5de-7de779f150cc None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:09.985060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae66fd-f54a-4d71-811b-9249e78161c5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:10.010349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.011602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.012083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.012557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.013537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] RBAC: Adding query filter params (interface=public, service_id=685152593ba34eb3b1004c49d5cb1fd3, region_id=RegionOne) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:10.014190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.016072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.024001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 474/937] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/endpoints?service_id=685152593ba34eb3b1004c49d5cb1fd3&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:10.032049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:10.059478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.060520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.061017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.061496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.075141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.077546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.087795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:10.101346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:10.102389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:10.103787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:10.103787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:10.103787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:10.110312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 464/938] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:10.112837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:10.149485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:10.234896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8622d7-22a2-4668-9277-0ee35ab271b1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:10.243423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cd8bb-ba1a-4aa4-abb2-2fe5752c205c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:10.299768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.300914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.301408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.302199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.303559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:10.304153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.304263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.305209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.305725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.306202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.306655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e233a-c327-4495-b9b7-607395a11096 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.307307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.310529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e349e40-d76d-4a63-8e90-4579042b048f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.322981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 475/939] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:10 2022] GET /identity/v3/services => generated 1497 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:37:10.328383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc952c-1db8-43c7-87ef-72fa8676e954 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.328383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc952c-1db8-43c7-87ef-72fa8676e954 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.328383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc952c-1db8-43c7-87ef-72fa8676e954 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.329261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 476/941] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:10.329487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 465/941] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:10.331607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc952c-1db8-43c7-87ef-72fa8676e954 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:10.336031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.336722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.337302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.360933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:10.362890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:10.486989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 466/942] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:10.543062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:10.552495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc952c-1db8-43c7-87ef-72fa8676e954 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:10.622288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.623359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.623833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.624318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.638007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.640797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:10.663956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:10.664940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:10.665876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:10.666869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 477/943] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:10.675831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:10.710938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:10.877490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f0832-06d0-4b82-83cd-3af3b8f171f3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:10.941161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:10.942185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:10.943220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:10.943688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:10.944871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:10.945335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:10.948122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:10.970314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 467/944] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:10.977269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:11.072079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:11.103598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208fb67-3f4a-40bb-90b5-3b4d2521d9be None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:11.161882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:11.163789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:11.164276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:11.164750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/2ab3501f173341538094ffeb2a59f004/roles/86c64cac25b34439972a071e46a61bca` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:11.169838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=2ab3501f173341538094ffeb2a59f004, role_id=86c64cac25b34439972a071e46a61bca)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:11.172678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:11.201193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 478/945] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:37:10 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/2ab3501f173341538094ffeb2a59f004/roles/86c64cac25b34439972a071e46a61bca => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:37:11.240880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:11.294346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e096b53-04d4-4e79-8e9b-920d3e9508bb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:11.360746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:11.361901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:11.362378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:11.362874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:11.363986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:11.367422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:11.386102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:11.387136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 11:37:11.388034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:11.388781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 468/946] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 6 11:37:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:11.394408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:11.424353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:11.573276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b22b8-651f-4384-b862-d29740716727 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:11.613528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:11.614709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:11.614992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:11.615262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:11.615916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:11.616218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:11.617631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:11.623593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 479/947] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:11.629131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:11.832277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a682691-5fbc-4799-a2f1-3aedc9038c05 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:11.901942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:11.902991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:11.903489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:11.903984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:11.905186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:11.905917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:11.908583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4f79d-6bc8-4734-92e5-91fc968eb091 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:11.919537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 469/948] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:11.924914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:12.020874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:12.046361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300f74b-2fbb-484a-af14-ba3733c6f39b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:12.117322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:12.118340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:12.118839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:12.119380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:12.120770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:12.123507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa422ec-7a21-4274-b5cc-eea1acfa5753 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:12.166956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 480/949] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:12.215556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f16b9-9268-4cc9-b7ea-746901f8880e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:12.216110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f16b9-9268-4cc9-b7ea-746901f8880e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:12.216520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f16b9-9268-4cc9-b7ea-746901f8880e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:12.219788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 470/950] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:12.229281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb3c-1d7f-498a-b107-867567a99242 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:12.230018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb3c-1d7f-498a-b107-867567a99242 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:12.230514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebb3c-1d7f-498a-b107-867567a99242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:12.267122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-acfebb3c-1d7f-498a-b107-867567a99242 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:12.268432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-acfebb3c-1d7f-498a-b107-867567a99242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:12.406302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 481/951] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:12.858622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c772-7bb6-487c-9d09-af4580460d2e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:12.859164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c772-7bb6-487c-9d09-af4580460d2e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:12.859660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c772-7bb6-487c-9d09-af4580460d2e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:12.862098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 471/952] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:12.870663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2459d7-715e-400a-948d-fd3b8f2abcda None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:12.870943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2459d7-715e-400a-948d-fd3b8f2abcda None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:12.871245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2459d7-715e-400a-948d-fd3b8f2abcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:12.893535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6f2459d7-715e-400a-948d-fd3b8f2abcda None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:12.894873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6f2459d7-715e-400a-948d-fd3b8f2abcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:12.949529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c772-7bb6-487c-9d09-af4580460d2e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:12.958989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:12.971031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:12.995575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 482/953] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:13.175259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c772-7bb6-487c-9d09-af4580460d2e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:13.247758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:13.248763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.249224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.249825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.251001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:13.253833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:13.280972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.sql.core [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 11:37:13.280972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 11:37:13.280972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [parameters: {'user_id': 'cc5e3f56a8de4e37b078308068fcfb31', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 11:37:13.280972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87996) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-022a21ad-ec40-4364-8304-ac3571b869f3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result.read() Sep 06 11:37:13.296419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:37:13.298214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application yield resource Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application next(self.gen) Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise self.value Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application session.commit() Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 11:37:13.300670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.session.flush() Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application compat.raise_( Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 11:37:13.305833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 11:37:13.309351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application conn.query(q) Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application result.read() Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 11:37:13.328740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cc5e3f56a8de4e37b078308068fcfb31', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:13.332107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 11:37:13.335300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.339179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 472/954] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 6 11:37:12 2022] POST /identity/v3/users => generated 160 bytes in 350 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:13.340322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9490af-7f92-419e-a105-d5673a2a8a6b None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.340322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9490af-7f92-419e-a105-d5673a2a8a6b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.340322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9490af-7f92-419e-a105-d5673a2a8a6b None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.340322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 483/955] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:13.341018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.341018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.341018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.341723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9490af-7f92-419e-a105-d5673a2a8a6b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:13.354970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:13.358277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:13.492964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:13.504530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 473/956] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:13.506544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:13.593542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9490af-7f92-419e-a105-d5673a2a8a6b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:13.661238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:13.662346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.662831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.663310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.678147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:13.679007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc408-0b7a-483d-a3c9-25a4389692da None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:13.681045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:13.704601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:13.714706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:13.716495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 484/957] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:37:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:13.720322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:13.769835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:13.772714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.773202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.773692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.774844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:13.778719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:13.790214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:13.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:13.792585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 474/958] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:13.793522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:13.804733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:13.872561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:13.902181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6694b6-6628-461d-bb7d-f681083f2900 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:13.937467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937b3d-b3bc-4ff0-bcbb-61b3f357d72d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:13.945927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:13.946809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.947427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.947873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.948685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:13.949120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:13.951340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:13.982714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:13.983431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:13.983707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:13.984015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:13.997960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:14.008767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 485/959] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 6 11:37:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:14.009371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:14.011300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:14.012870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:14.022619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.022735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 475/960] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:14.031361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.231564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb877-8c7d-4cef-ae67-a20a6fb7e5f9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.239104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece95b-3d30-4b3c-a68f-40fbbc321758 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.253487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:14.293776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:14.304919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.306207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.306825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.307427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.308691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:14.309359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.312158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.322890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 486/961] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:14.329993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.334349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.335429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.335766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.336002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.336932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:14.337310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.339866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.355507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 476/962] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:14.363650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.531702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747e2be-2ed1-4232-9902-5adb5da2d72e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.540273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c32ff-2e8f-42bd-8ad3-89e73cd20928 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.588262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.588927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.589093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.589352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.603589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.605080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.606661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.607644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.608591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.608591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.609751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:14.610258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.612866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.622732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 487/963] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:14.624400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:14.626301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:14.628179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:14.628179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:14.628179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:14.630213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 477/964] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:14.632127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.632770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:14.690108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:14.701082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:14.797129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b3bf-534f-480d-80d6-694ce98892e6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.834853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cab5f1-0bb9-433e-9c13-2b78562cef6b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:14.844726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.845259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.845541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.845831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.846555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.848000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f7561-86cc-475b-9d73-99ad7677de94 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.879572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 488/965] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:14.883762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb8a8-6931-442c-a28d-5019b123c1a9 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.884183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb8a8-6931-442c-a28d-5019b123c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.884557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb8a8-6931-442c-a28d-5019b123c1a9 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.887667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 489/966] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:14.893018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.893396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.893776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.918696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:14.920385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:14.923716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:14.924920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:14.925514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:14.926047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:14.927543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:14.928097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:14.930895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:14.960498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 478/967] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:14.962576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.027114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 490/968] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:15.111031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:15.126286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4543-3690-4497-a41b-83d0143f00de None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:15.171232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:15.171896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:15.172176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:15.172449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:15.181844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:15.183352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:15.197717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:15.199475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:15.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 479/969] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:37:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:15.204976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.255338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:15.315783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.405005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e089e-9bbb-4e9c-92f4-81629d0d8475 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:15.461787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:15.462616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:15.463068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:15.463521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:15.464782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:15.465286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:15.467894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:15.468159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5093fc6-7223-44ee-8ad9-839c6cd22b42 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:15.497328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 491/970] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 6 11:37:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:15.504051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.539237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:15.540283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:15.540745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:15.541202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:15.542258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:15.545049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:15.562657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:15.564628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:15.566258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 480/971] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:15.572867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.603976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:15.642295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:15.671498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02c2fc-27c4-4699-9ded-4299c3b1118e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:15.716089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:15.716886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:15.717205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:15.717606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:15.733194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4115d-ce15-4c67-9118-f43a71695fd3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:15.791003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:15.791870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:15.792310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:15.792750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:15.793588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:15.794023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:15.796438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:15.810873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 481/972] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:15.812747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:15.831922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:15.835959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:15.949085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:15.980644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:16.005903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1e3d-6cc3-4ff4-b1a7-9b2d1ce14bc5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:16.025916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 492/973] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:37:16.028393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:16.078879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:16.079691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:16.080070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:16.080447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:16.081367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:16.081925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:16.085779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:16.101682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 482/974] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:16.235992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19246da-a060-4440-8804-e1b5a61f3a20 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:16.322395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:16.323516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:16.324006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:16.324495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:16.340313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:16.342004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:16.361530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:16.363415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:16.365577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:16.365577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:16.365577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:16.365577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 493/975] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:16.373191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:16.554742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef1258-3333-4c4e-b189-479292388c36 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:16.620911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:16.622012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:16.622492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:16.622976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:16.624173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:16.624718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:16.627239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:16.654903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 483/976] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:16.657361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:16.717692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:16.839059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085452e-f5e5-4277-b60b-a88f0837b979 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:16.882632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:16.883281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:16.883550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:16.883814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:16.892521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:16.894131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:16.908155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:16.909961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:16.912421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 494/977] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:16.920889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:16.925590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:17.086757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d42da0-ac4f-4fb5-b591-526e333583af None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:17.128064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:17.128585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:17.128859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:17.129123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:17.130374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:17.130674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:17.132359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989023ad-e4b5-49da-b149-67f4cccd9d61 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:17.148291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 484/978] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:17.154215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:17.281613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a65ae-6f2f-4097-948b-4455b1edb6a1 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:17.282240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a65ae-6f2f-4097-948b-4455b1edb6a1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:17.282551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a65ae-6f2f-4097-948b-4455b1edb6a1 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:17.284281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 485/979] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:17.291458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:17.292340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:17.292622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:17.292891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:17.304056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023a45-96b0-4266-84c2-1431ddbbbc1f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:17.313495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:17.314784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:17.349062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:17.349719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:17.350002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:17.350284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:17.351082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] RBAC: Adding query filter params (role.id=86c64cac25b34439972a071e46a61bca, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=2ab3501f173341538094ffeb2a59f004) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:17.351422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:17.352905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:17.360099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 495/980] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:17 2022] GET /identity/v3/role_assignments?user.id=2ab3501f173341538094ffeb2a59f004&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=86c64cac25b34439972a071e46a61bca => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:17.389848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:17.407506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 486/981] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:17.694380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:17.824959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b3fc-a2aa-45ac-a824-a544eb44442b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:17.869037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:17.869716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:17.870008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:17.870257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:17.870849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:17.872780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:17.887447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:17.888611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:17.889627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:17.890606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 496/982] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:17.897956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:18.026463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2deede-c161-4426-8d42-9fe728d86585 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:18.070166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:18.070786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:18.071061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:18.071330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:18.072003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:18.072308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:18.073767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:18.079938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 487/983] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:18.086971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:18.131535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:18.268119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efa1cc-d84d-4c10-a1ed-1025b6fae4f7 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:18.312986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4869-2124-4658-882f-307bf783f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:18.313899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4869-2124-4658-882f-307bf783f170 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:18.314322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4869-2124-4658-882f-307bf783f170 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:18.314765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4869-2124-4658-882f-307bf783f170 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:18.315571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b4869-2124-4658-882f-307bf783f170 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:18.316032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b4869-2124-4658-882f-307bf783f170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:18.318357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b4869-2124-4658-882f-307bf783f170 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:18.324785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 497/984] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:18.332335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:18.407720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:18.475164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71a40-5a09-431d-a061-38c29b3f56d6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:18.516009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:18.516649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:18.516929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:18.517201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:18.517999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] RBAC: Adding query filter params (interface=admin, service_id=685152593ba34eb3b1004c49d5cb1fd3, region_id=RegionOne) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:18.518314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:18.519762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795435da-8fc5-4558-a658-3bd3cc1f0042 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:18.529753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 488/985] 10.222.0.18 () {56 vars in 1249 bytes} [Tue Sep 6 11:37:18 2022] GET /identity/v3/endpoints?service_id=685152593ba34eb3b1004c49d5cb1fd3&interface=admin®ion_id=RegionOne => generated 193 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:18.532428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4869-2124-4658-882f-307bf783f170 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:18.644237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:18.665184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4869-2124-4658-882f-307bf783f170 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:18.704623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:18.705522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:18.706002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:18.706416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:18.707188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:18.708804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af38160-5e71-4625-a63e-7ae1ce323acb None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:18.721811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 498/986] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:18 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:37:19.279479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3d43-6ccc-43e4-9418-3039f48a5477 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:19.279949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3d43-6ccc-43e4-9418-3039f48a5477 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:19.280420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3d43-6ccc-43e4-9418-3039f48a5477 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:19.283006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 489/987] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:19.292064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464698b1-a66a-448d-9338-dc22ff9ba686 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:19.292643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464698b1-a66a-448d-9338-dc22ff9ba686 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:19.292643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464698b1-a66a-448d-9338-dc22ff9ba686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:19.325919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-464698b1-a66a-448d-9338-dc22ff9ba686 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:19.327856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-464698b1-a66a-448d-9338-dc22ff9ba686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:19.430852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 499/988] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:19.507117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ff312-ef67-4e03-8dee-a79b91c795dc None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:19.507117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ff312-ef67-4e03-8dee-a79b91c795dc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:19.507117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ff312-ef67-4e03-8dee-a79b91c795dc None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:19.509158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 490/989] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:19.515172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:19.515473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:19.515797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:19.543202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:19.545359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:19.676143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 500/990] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:19.717530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ff312-ef67-4e03-8dee-a79b91c795dc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:19.838722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ff312-ef67-4e03-8dee-a79b91c795dc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:19.865477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:19.883535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:19.884168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:19.884443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:19.884713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:19.885391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:19.887091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:19.910130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 491/991] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:19 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:37:20.203321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:20.227740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:20.414387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f86e3-7df9-4d80-9492-a28675dde765 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:20.501156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:20.502227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:20.502728 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:20.503230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] PATH_INFO: `/v3/services/image` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:20.504356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:20.507705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:20.527497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:20.529400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:20.531607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 501/992] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 6 11:37:20 2022] GET /identity/v3/services/image => generated 86 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:20.535572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:20.673373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b207e1-1bcb-4ba2-a6e0-84222fed2c93 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:20.715813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:20.716473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:20.716860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:20.717266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:20.717895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] RBAC: Adding query filter params (name=image) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:20.718201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:20.719645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98489a56-4a35-4907-a1ea-c21f7e34eac5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:20.726973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 492/993] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:20.733397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:20.863789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:20.935788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e37e2c-f1c3-4dfd-a854-d8eda5541286 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:21.006278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:21.007726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:21.008213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:21.008737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:21.010127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] RBAC: Adding query filter params (type=image) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:21.010934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:21.013708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159a69b-11e4-4c35-a966-b57a77d78e50 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:21.030008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 502/994] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:21.077638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44c8c-0b8e-4141-aa0b-c4253c6838d6 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:21.077995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44c8c-0b8e-4141-aa0b-c4253c6838d6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:21.078262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44c8c-0b8e-4141-aa0b-c4253c6838d6 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:21.079972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 493/995] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:21.086002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:21.086271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:21.086636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:21.108326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:21.109631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:21.202525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 503/996] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:21.484758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44c8c-0b8e-4141-aa0b-c4253c6838d6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:21.587752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:21.695593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44c8c-0b8e-4141-aa0b-c4253c6838d6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:21.764889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:21.765872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:21.766345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:21.766798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:21.767864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:21.770702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:21.789282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:21.791434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:21.793471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 494/997] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 6 11:37:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:21.797591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:21.877281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:21.928490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411da9d-d1ce-48be-8c81-3bf15d802cc3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:21.973257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:21.974238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:21.974238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:21.974739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:21.975280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:21.975581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:21.977012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb8065-142e-4b2c-996f-f4487a1c47ec None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:21.988364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 504/998] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:21.990744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:22.046065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc31ca3-b48d-4611-bfc1-56c154352eae None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.046065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc31ca3-b48d-4611-bfc1-56c154352eae None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.046433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc31ca3-b48d-4611-bfc1-56c154352eae None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.047981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 505/999] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:22.054846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.055102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.055477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.076633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:22.077908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:22.122662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a64af-0d12-419f-a36a-ac504927e0f6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:22.134840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:22.166319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 506/1000] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:22.172490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:22.173316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.173843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.174240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.175062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:22.175501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:22.177328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:22.188759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 495/1001] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 6 11:37:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:22.190047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:22.273439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:22.331298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb910d3-e541-4a5f-bb67-6f29f6f34181 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:22.375659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:22.376269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.376546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.376820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.377529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:22.378975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:22.405063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 507/1002] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:22.469228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:22.610993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e185-89ee-4284-828a-232e8a2a7a36 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:22.668098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:22.668919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.669326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.669792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.670538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:22.672206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:22.684091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:22.686655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:22.688338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 496/1003] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:22.694656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:22.829377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e1a6c-368f-48c3-a60d-c04bc74202f4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:22.850507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:22.886473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:22.887433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:22.887892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:22.888361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:22.889599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:22.890233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:22.892846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:22.903612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 508/1004] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:22.913779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:22.978659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:23.127496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6f46-9dfa-403a-a5af-d5e16daec0e5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:23.197944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:23.198986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:23.199477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:23.199956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:23.201098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:23.201623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:23.204246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c7ca2-0f90-416d-b3a1-7c0aeef6ad3d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:23.214266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 497/1005] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:23.224384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:23.404359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41020d7-6f42-432e-8990-9abe82533363 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:23.404842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41020d7-6f42-432e-8990-9abe82533363 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:23.405306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41020d7-6f42-432e-8990-9abe82533363 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:23.407887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 498/1006] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:23.417310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:23.417772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:23.418213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:23.441188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9bbb-427a-4595-bbdd-b9e59931b153 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:23.452243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:23.454192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:23.490336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:23.490975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:23.491281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:23.491525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:23.492235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] RBAC: Adding query filter params (interface=public, service_id=0a697ed33be7425d999c2640764a5743, region_id=RegionOne) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:23.492541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:23.494025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:23.501230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 509/1007] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:37:23 2022] GET /identity/v3/endpoints?service_id=0a697ed33be7425d999c2640764a5743&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:23.512105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:23.574736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 499/1008] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:23.608039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:23.730432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169aa89-72f2-40d9-9685-4cdeacab6880 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:23.801153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:23.803035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:23.803467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:23.803899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:23.804933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:23.807515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a6111-756d-4b94-a783-dd9c7ee24f1b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:23.825992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 510/1009] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:23 2022] GET /identity/v3/services => generated 1727 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:37:23.916979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:23.953709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:24.069453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff48c0-3f4e-4003-be20-ff2921f6baee None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:24.129383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:24.130227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:24.130646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:24.131222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:24.131774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:24.133302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dfdab-4f81-4b45-95fc-9d4a9b0d7353 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:24.159427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 500/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:23 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:37:24.595319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59fdf5-c067-4f66-9d16-463a58364849 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:24.595925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59fdf5-c067-4f66-9d16-463a58364849 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:24.596328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59fdf5-c067-4f66-9d16-463a58364849 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:24.599570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 511/1011] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:24.605727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:24.606011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:24.606332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:24.628980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:24.630933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:24.743379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 501/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:24 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:25.042944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59fdf5-c067-4f66-9d16-463a58364849 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:25.174282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59fdf5-c067-4f66-9d16-463a58364849 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:25.187812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:25.222978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:25.224224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:25.224224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:25.224851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:25.225936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:25.227401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee07440-a031-4996-985a-66e60e2761d6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:25.229252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 512/1013] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:25.235042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:25.253898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:25.362839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75b204-8e33-4bb7-9797-ad029b695d7c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:25.401869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:25.402497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:25.402772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:25.403040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:25.403715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:25.405185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e88130-bd4e-48d6-9748-53a8a3432636 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:25.440690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 502/1014] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:25 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:26.575502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97320717-c104-4b13-844a-63dfbd904f56 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:26.576239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97320717-c104-4b13-844a-63dfbd904f56 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:26.576239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97320717-c104-4b13-844a-63dfbd904f56 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:26.577329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 513/1015] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:26.584149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a007-39d9-4fb1-ac2b-50297e0f3a33 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:26.584957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a007-39d9-4fb1-ac2b-50297e0f3a33 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:26.584957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a007-39d9-4fb1-ac2b-50297e0f3a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:26.605034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-75a3a007-39d9-4fb1-ac2b-50297e0f3a33 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:26.606320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-75a3a007-39d9-4fb1-ac2b-50297e0f3a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:26.686692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:26.701751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 503/1016] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:26.774978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127949d0-59e9-494a-a855-fe9367226678 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:26.775519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127949d0-59e9-494a-a855-fe9367226678 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:26.775987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127949d0-59e9-494a-a855-fe9367226678 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:26.778582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 514/1017] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:26.784987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:26.785292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:26.785522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:26.807246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:26.808468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:26.893159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 504/1018] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:26.982475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127949d0-59e9-494a-a855-fe9367226678 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.111163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127949d0-59e9-494a-a855-fe9367226678 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.156271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.157130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.157573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.158073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] PATH_INFO: `/v3/services/image` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.158903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.160523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:27.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:27.173607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:27.175984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.179922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 515/1019] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 6 11:37:26 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:27.181028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.181297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.250880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:27.321806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93599b91-152c-4f20-89c3-16d33e60f7ac None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.352129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:27.366053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32cf4-98e1-4281-b3f5-3466310cf7c1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.366360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.366782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.367060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.367314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.367974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] RBAC: Adding query filter params (name=image) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:27.368273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.369859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.382710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 505/1020] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:27.389211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.432582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.433340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.433624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.434061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.434674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.436093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:27.446171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:27.447271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:27.448343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 516/1021] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:27.453895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.574970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90572ed-03bf-42f0-980f-3b26c3d1f359 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.577891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:27.604656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b300-ba18-4837-821c-7a19b9d4701a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.619561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.620273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.620558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.620829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.621497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:27.621860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.623431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.634368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 517/1022] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:27.636534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.673600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.674680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.675164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.675652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.677493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] RBAC: Adding query filter params (type=image) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:27.678152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.680778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.697393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 506/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:27.702722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.805503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f68c40-d055-431b-9875-c00be28aa1fe None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.807651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:27.862472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.863240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.863611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.864630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.865491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:27.865953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.867515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8054b0-96c8-43fd-9277-730cdd2c5c68 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.873313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 518/1024] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:27.874885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b448-d8ac-4df5-be4a-72cc69e54487 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.875159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b448-d8ac-4df5-be4a-72cc69e54487 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.875429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b448-d8ac-4df5-be4a-72cc69e54487 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.877031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 519/1025] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:27.880941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63b448-d8ac-4df5-be4a-72cc69e54487 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:27.886848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f073cf0-703e-4d96-bac4-bdbe6be4716c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:27.917827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:27.938308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:27.939240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.939609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.940068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.941004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] RBAC: Adding query filter params (interface=public, service_id=1b94e9a31ff3401982f9dd1bee224bff, region_id=RegionOne) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:27.941360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:27.943075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55a0bf-8fff-4487-a345-ddd6f0abb912 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:27.949552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 507/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:37:27 2022] GET /identity/v3/endpoints?service_id=1b94e9a31ff3401982f9dd1bee224bff&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:27.951393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:27.951839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:27.952170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:27.972577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:27.973936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:28.021634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63b448-d8ac-4df5-be4a-72cc69e54487 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:28.065193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:28.066030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:28.066372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:28.066719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:28.067519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:28.069375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:28.089839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 508/1027] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:28.091174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:28.108400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 520/1028] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:37:28.163264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:28.244530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abd611-8440-49ef-bb85-5fa0911b1fad None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:28.307764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:28.308461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:28.308720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:28.308983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:28.309702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:28.311174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:28.320350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 509/1029] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:37:28 2022] GET /identity/v3/services => generated 1957 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:37:28.475695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:28.640333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:28.670290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25631-707b-43dd-9b18-778f03173912 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:28.745771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:28.746865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:28.747355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:28.747847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:28.750046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:28.752838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473189f0-9817-4d55-86c7-1c971ebd4082 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:28.757712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 521/1030] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:28.764017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:28.958671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850538-8ba1-458d-953c-954062deb948 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:28.975191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:29.033604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:29.034803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:29.035381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:29.036474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:29.037772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:29.040612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c88ce-d775-4d96-b15f-0bec15237c69 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:29.087738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 510/1031] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:37:28 2022] POST /identity/v3/users => generated 285 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:30.347540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652076-3261-4feb-9da5-1b315f522041 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:30.348974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652076-3261-4feb-9da5-1b315f522041 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:30.348974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652076-3261-4feb-9da5-1b315f522041 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:30.353701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 522/1032] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:30.358309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e198-c9d8-47c1-b4ee-821a63e2a895 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:30.358846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e198-c9d8-47c1-b4ee-821a63e2a895 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:30.358846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e198-c9d8-47c1-b4ee-821a63e2a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:30.378866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-74e3e198-c9d8-47c1-b4ee-821a63e2a895 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:30.380144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-74e3e198-c9d8-47c1-b4ee-821a63e2a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:30.467434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 511/1033] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:30.523291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8ceca-561d-4f22-8572-532039706094 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:30.523684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8ceca-561d-4f22-8572-532039706094 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:30.524180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8ceca-561d-4f22-8572-532039706094 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:30.526959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 523/1034] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:30.537248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:30.537789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:30.538294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:30.568722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:30.570011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:30.657059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 512/1035] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:30.747969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8ceca-561d-4f22-8572-532039706094 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:30.877407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8ceca-561d-4f22-8572-532039706094 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:30.919560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:30.920198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:30.920471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:30.920741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] PATH_INFO: `/v3/services/image` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:30.921332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:30.922777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:30.932687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:30.934609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 11:37:30.935538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:30.938045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 524/1036] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 6 11:37:30 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:30.939812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:30.944049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:30.971835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:30.978667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:31.077613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e261f4-adab-4e3d-a6b4-871ac8d877c4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.119379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.120004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.120278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.120547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.121191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] RBAC: Adding query filter params (name=image) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:31.121354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3da9-356d-4176-8502-aefb3f06930e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.121523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:31.122924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:31.130241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 513/1037] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:31.136599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:31.188582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.189571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.190069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.190501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.214032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:31.224708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:31.226253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:31.227521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 525/1038] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:31.235189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:31.266563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:31.291354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a125b-66ac-4b12-a35d-771b7880c81a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.353475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.354136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.354416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.354686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.355339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] RBAC: Adding query filter params (type=image) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:31.355656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:31.357207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a297251-b74d-4c20-8e4f-bc392dddaeb9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:31.362808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 514/1039] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:31.364162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86c401-b929-4c74-b2f1-33957907edc2 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.364517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86c401-b929-4c74-b2f1-33957907edc2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.364775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86c401-b929-4c74-b2f1-33957907edc2 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.366339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 515/1040] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:31.370830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86c401-b929-4c74-b2f1-33957907edc2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:31.377557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:31.410583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d8651-3a10-4c66-b206-6185db4eec95 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.467859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.468832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.469317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.469795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.470697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:31.471194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:31.474183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22c9c-5fb4-4aa2-b94f-cadc73ee8cf3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:31.484257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 526/1041] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:31.485527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.485838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.486112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.506107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:31.507357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:31.534730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86c401-b929-4c74-b2f1-33957907edc2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.575949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.576582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.576859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.577150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.577841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:31.579254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:31.593084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 527/1042] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:31.594003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:31.611969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 516/1043] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:37:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:37:31.735995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:31.805849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740fdd-bc5e-46ea-a97f-a52bd3c379cf None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:31.878558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:31.879994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:31.880686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:31.881384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:31.888250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:31.899422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:31.901691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:31.905552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:31.922252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:31.924030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:31.925897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:31.925897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:31.925897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:31.925897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 528/1044] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:31.930319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.078606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf14a-618c-4679-bb14-53724f26b456 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.121412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.122110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.122395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.122667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.135192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9a86a-7211-412d-9e43-c06ab2fcd34b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.137847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:32.144000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:32.146014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:32.147728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:32.152427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 517/1045] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:32.157009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.183370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:32.211188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.211867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.212184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.212487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.213394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:32.213777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:32.215574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:32.231709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 529/1046] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:32.237802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.277735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:32.350846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f607dc2-5d56-4866-bea2-bc883bb5df16 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.402997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2ca51-8521-4546-8c40-3e3fa9b418ea None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.416492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.417315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.417805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.418207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.419137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:32.419469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:32.421915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:32.434321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 518/1047] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:32.443050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.452485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.453522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.454047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.454518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.464076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:32.466009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:32.481572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:32.490660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 530/1048] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:32.492447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.545909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:32.638818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38061a-91a9-46cf-ba17-3620b9d35264 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.651301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f15b31-1d93-41f3-b7db-82ab2f9154f6 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.696890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.697537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.697850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.698126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.698914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:32.699219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:32.700840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:32.702303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:32.720114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.720114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.720114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.720114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] PATH_INFO: `/v3/users/baae6caa7c274a62bbb25825b82a4c4f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:32.731690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 531/1049] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:32.738803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.749777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] RBAC: Authorizing `identity:get_user(user_id=baae6caa7c274a62bbb25825b82a4c4f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:32.752639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:32.756192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 519/1050] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/users/baae6caa7c274a62bbb25825b82a4c4f => generated 285 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:32.764754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:32.903589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1dff-c108-48cc-88a3-23bb621a3088 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.936108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:32.943350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a502c2-64d3-43b6-bdcb-9688c47d6c60 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:32.947528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:32.948311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:32.949959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:32.949959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.001324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:33.002392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.002877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.003352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.050213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:33.077294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.080163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.098129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.100607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.244799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:33.266006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 520/1051] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:37:33.266277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 532/1052] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:37:33.267883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:33.268138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:33.410732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158eb9e-9a43-4430-b183-0a5e2a1dd66c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:33.433924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:33.460125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1c578-a151-4e18-91a0-44513f28fbcf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:33.464117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:33.464828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.465137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.465490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.467243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.469803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.473866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 533/1053] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:33 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:33.485593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:33.526696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:33.527589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.528030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.528598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.546781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.549634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:33.580460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:33.582299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:33.585608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:33.585608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:33.585608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:33.585608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 521/1054] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:33.588799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:33.610362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:33.664988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7339fb-7272-4be6-9d4d-9e5080f2a01c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:33.733390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:33.734151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.734494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.734778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.735534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06, user.id=baae6caa7c274a62bbb25825b82a4c4f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:33.735837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.737304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46310a7b-3b2e-4588-aa97-a2f74d106b0c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.743879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 534/1055] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:33 2022] GET /identity/v3/role_assignments?user.id=baae6caa7c274a62bbb25825b82a4c4f&scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:33.755581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdbd76-9484-4e75-814b-084a80d8ce68 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.755893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdbd76-9484-4e75-814b-084a80d8ce68 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.756134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdbd76-9484-4e75-814b-084a80d8ce68 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.757599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 535/1056] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:33.767090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.767379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.767631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.791291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a018-88ae-47d5-9b43-80cd59ef13dd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:33.791792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:33.793099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:33.855504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:33.856715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:33.857198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:33.857688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:33.858959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:33.859493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:33.862161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:33.878195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:33.888284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 522/1057] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:33.896219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:33.908022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 536/1058] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:34.072066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ae78-4b25-497c-be62-e19d3ca723b2 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:34.118225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:34.118877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:34.119147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:34.119420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:34.128103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:34.130656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:34.144029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:34.146585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:34.148338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 523/1059] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:34.154081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:34.201721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:34.229485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:34.314251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f254-ec6a-46a2-a2a3-86544ba1f102 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:34.324160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:34.378466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:34.379131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:34.379944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:34.380226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:34.381524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:34.381524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:34.382767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:34.394297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56fcb3-8414-4f56-8a6a-06f8dece73db None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:34.398652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 537/1060] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:34.404766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:34.469004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:34.470275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:34.470821 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:34.471299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:34.576569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:34.577098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd1f7c-8e3c-4f78-9d4b-3744d26ef38a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:34.579345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:34.610033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:34.637841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:34.638917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:34.639415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:34.639895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:34.641270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] RBAC: Adding query filter params (role.id=86c64cac25b34439972a071e46a61bca, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:34.642895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:34.644672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:34.665695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 538/1061] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:34 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=86c64cac25b34439972a071e46a61bca => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:34.684024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:34.746377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 524/1062] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:37:34.753863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:34.915151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebd19-ccf2-4eec-90bf-62e9618babf9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:34.958509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:34.959154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:34.959434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:34.959693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:34.974098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:34.975568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-327a609b-ff1d-4451-ba33-c3a5ef31de17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:34.993305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:34.995541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:34.997392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:34.997392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:34.997392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:35.000025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 539/1063] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:35.002105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:35.135155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b906ea-17fc-4530-927d-026cf702f27a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:35.191507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:35.193034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:35.193763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:35.194230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:35.195444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:35.195972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:35.198518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53071dc9-0702-404b-9dbe-ea18d066b471 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:35.221703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 525/1064] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:35.883924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486591f7-29d2-4488-b79e-97912c9b74e9 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:35.884516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486591f7-29d2-4488-b79e-97912c9b74e9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:35.884853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486591f7-29d2-4488-b79e-97912c9b74e9 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:35.887687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 540/1065] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:35.895797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:35.896329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:35.896329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:35.916468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:35.917748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:35.936724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:36.027249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 526/1066] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:36.302670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486591f7-29d2-4488-b79e-97912c9b74e9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:36.346464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:36.424641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486591f7-29d2-4488-b79e-97912c9b74e9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:36.466787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:36.467421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:36.467698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:36.467970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:36.480634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:36.485577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:36.486863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:36.488049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 541/1067] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:37:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:36.495090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:36.634984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312bc71-348d-4930-84e3-1bb1f38120fe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:36.677722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:36.678353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:36.678626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:36.678895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:36.679587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:36.679964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:36.681457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:36.691410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 527/1068] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:37:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:36.698484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:36.827748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:36.836258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00572-5a90-41cf-9e81-a1e27018ccfb None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:36.880800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:36.881564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:36.881971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:36.882333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] PATH_INFO: `/v3/users/baae6caa7c274a62bbb25825b82a4c4f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:36.883659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] RBAC: Authorizing `identity:get_user(user_id=baae6caa7c274a62bbb25825b82a4c4f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:36.885388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:36.887772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 542/1069] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:36 2022] GET /identity/v3/users/baae6caa7c274a62bbb25825b82a4c4f => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:36.893909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:36.909335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:37.022455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fedc8-97f7-4a4a-aa9f-88e4d4afa508 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:37.061304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:37.061986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.062267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.062544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.063762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:37.065232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5d80-e249-4cc8-8591-346953fe6fa6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:37.067693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 528/1070] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:36 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:37.078015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:37.122552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8607519-c90d-47c9-88e2-dbb73515058a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.122841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8607519-c90d-47c9-88e2-dbb73515058a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.123215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8607519-c90d-47c9-88e2-dbb73515058a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.125167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 529/1071] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:37.131577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.131861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.132136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.152063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:37.153203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:37.203892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c34a5-34a1-49f3-a0f8-5c71eb932e1a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:37.247030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 530/1072] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:37.256114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:37.256799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.257075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.257348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06/users/baae6caa7c274a62bbb25825b82a4c4f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.259521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4dc8e171e884e31bc7b1546ee7bfd06, user_id=baae6caa7c274a62bbb25825b82a4c4f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:37.261244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:37.277152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 543/1073] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:37:37 2022] PUT /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06/users/baae6caa7c274a62bbb25825b82a4c4f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:37:37.527220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:37.539759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:37.649246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb9d31-80f6-45df-8d1c-84dc0e657eee None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:37.691312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:37.692072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.692350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.692621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.704991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:37.710583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:37.712460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:37.714165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 531/1074] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:37.722039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:37.770473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:37.862467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a6390-bdbc-4d94-96cb-ad3e4af24bf4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:37.902982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:37.903718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:37.903990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:37.904336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:37.905068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:37.905424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:37.907077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:37.917214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 544/1075] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:37.923117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:38.021531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:38.086986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88213650-255a-42c7-ae55-3d68bb334772 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:38.133820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:38.134347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:38.134616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:38.134884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:38.148824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:38.150346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:38.166884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:38.168963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:38.170922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:38.170922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:38.170922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:38.170922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 532/1076] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:38.178002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:38.291736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:38.314852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe105c-5319-4cb8-acf0-0cdeb80e1523 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:38.386430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:38.387306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:38.387769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:38.388217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:38.389394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:38.389956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:38.392405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:38.416403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 545/1077] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:38.424391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:38.568483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19cb40-39d4-4264-b8a7-e875b1bfc812 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:38.591995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:38.611102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:38.611743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:38.612012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:38.612279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] PATH_INFO: `/v3/users/nova` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:38.626063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:38.627553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:38.641076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:38.643055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:38.645595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:38.647573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 533/1078] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:37:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:38.649015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:38.763749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447594-0f60-4e42-98a9-8c0491241e09 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:38.805269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:38.805951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:38.806229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:38.806499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:38.807243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:38.807548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:38.809013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:38.828739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 546/1079] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:37:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:38.836344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:38.992532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ac285-14e8-40c1-a019-81573248392d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:39.059148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:39.060213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.060723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.061203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.083800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:39.086374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c9748a7c-c538-4942-be03-7ecff1b6ea90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:39.115141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:39.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:39.118857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:39.118857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:39.118857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:39.118857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 534/1080] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:39.124761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:39.129094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:39.313602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8c72-c765-4862-9435-2cd74c1dbde6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:39.379497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:39.380561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.381048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.381526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.382781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:39.383337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:39.386257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:39.409542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 547/1081] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:39.412976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4d1f-905a-419f-85eb-9495a7b2e9cb None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.413482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4d1f-905a-419f-85eb-9495a7b2e9cb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.414018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4d1f-905a-419f-85eb-9495a7b2e9cb None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.415644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:39.416744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 535/1082] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:39.423434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.424010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.424413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.456880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:39.459278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:39.509362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:39.511047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:39.564140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8b5e-c17e-4bf0-96aa-c8a12fb99736 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:39.576183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 536/1083] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:37:39.602409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:39.603069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.603347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.603618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.611663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:39.613142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:39.626170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:39.627581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:39.628687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 548/1084] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:39.634524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:39.758655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c307a-8c3b-433f-a090-6eed803cff6f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:39.802168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:39.802829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:39.803191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:39.803523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:39.804317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:39.804688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:39.806297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:39.821446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 537/1085] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:39.828571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:39.910264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:39.942274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:39.960682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a75c25-fbe6-41eb-b9ef-6dbbcd2b1a61 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:40.001625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:40.024022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:40.025101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:40.025539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:40.026060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/86c64cac25b34439972a071e46a61bca` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:40.029822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=9260fd9f9c524d20a9025595a53ade4e, role_id=86c64cac25b34439972a071e46a61bca)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:40.032591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:40.040598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887e323-6aa7-43a7-bd0d-0e971e15c6d1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:40.050173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 549/1086] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:37:39 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/9260fd9f9c524d20a9025595a53ade4e/roles/86c64cac25b34439972a071e46a61bca => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:37:40.087739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:40.088293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:40.088565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:40.088832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:40.103415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:40.109338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:40.111451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:37:40.113171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:40.115778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 538/1087] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:40.118600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:40.309508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f9bb4-9564-455f-a45d-dab62baa0266 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:40.362585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:40.363112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:40.363390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:40.363757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:40.365194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:40.365499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:40.366963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:40.377413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 550/1088] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:40.385500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:40.556038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:40.587197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bd27d-91a8-4a0c-97de-59117afbd43a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:40.655561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:40.656424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:40.656702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:40.656972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] PATH_INFO: `/v3/users/baae6caa7c274a62bbb25825b82a4c4f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:40.659162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] RBAC: Authorizing `identity:get_user(user_id=baae6caa7c274a62bbb25825b82a4c4f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:40.662130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:40.672472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 539/1089] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:40 2022] GET /identity/v3/users/baae6caa7c274a62bbb25825b82a4c4f => generated 285 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:40.674012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:40.736063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:40.811482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa404-26bc-45f2-89fd-f9d9d8348f79 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:40.856670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:40.857416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:40.857836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:40.858198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:40.946598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:40.948206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:41.007569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:41.077078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 551/1090] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:37:41.083983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:41.232943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c395-3639-4a7a-b123-03a110171212 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:41.293980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:41.295096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:41.295601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:41.296108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:41.298408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:41.301924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c905aa-bf4b-4696-a011-30cf400e14d0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:41.310942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 540/1091] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:41 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:41.312400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:41.436332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19fd9-db8b-4322-af21-e1ce809cceb1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:41.478363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:41.478979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:41.479251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:41.479517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:41.480258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06, user.id=baae6caa7c274a62bbb25825b82a4c4f) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:41.480559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:41.482074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9b59-21fd-4142-a7e9-da3984c61559 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:41.490005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 552/1092] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:41 2022] GET /identity/v3/role_assignments?user.id=baae6caa7c274a62bbb25825b82a4c4f&scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:42.178395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68b771-78d4-48b7-b37c-bcf7d18fc275 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:42.178986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68b771-78d4-48b7-b37c-bcf7d18fc275 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:42.179326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68b771-78d4-48b7-b37c-bcf7d18fc275 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:42.181903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 541/1093] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:42.190243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:42.190707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:42.191079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:42.213061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:42.214338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:42.217878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:42.309731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 553/1094] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:42.598909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68b771-78d4-48b7-b37c-bcf7d18fc275 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:42.632354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:42.790639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68b771-78d4-48b7-b37c-bcf7d18fc275 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:42.862618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:42.864166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:42.865765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:42.866269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:42.890995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:37:42.902883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:42.904610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:42.906298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 542/1095] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 6 11:37:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:42.913316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:43.038601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2ad65-4137-49fc-8983-c894c7294e39 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:43.080017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:43.080640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.080908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.081170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.081888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:43.082202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:43.084269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:43.094945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 554/1096] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:37:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:43.102247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:43.209351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:43.238128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fda-5b03-4252-9364-7e945664bce5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:43.302686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:43.303759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.304240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.304703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.323796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:43.326650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:43.353153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:43.355147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:43.357038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:43.357038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:43.357038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:43.357038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 543/1097] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:43.362655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:43.366960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:43.491361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac1728-d462-45d7-8bee-890aff104c35 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:43.536020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:43.536648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.536926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.537327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.537899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:43.538204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:43.539622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ad3e1-a12f-4cdd-ac82-aa2caf6ea910 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:43.556920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 555/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:43.565317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:43.660959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2b827-adf3-487d-9051-f45500f2afbc None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.661416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2b827-adf3-487d-9051-f45500f2afbc None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.661916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2b827-adf3-487d-9051-f45500f2afbc None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.664409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 556/1099] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:37:43.671499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.671971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.672352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.690061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea100108-d43c-4b5b-b70d-c3d835460513 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:43.703535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:43.705395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:43.728174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:43.735214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:43.735864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.736145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.736453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.747331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:43.748782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:37:43.762534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:43.764547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:43.766585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 544/1100] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 6 11:37:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:43.770884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:43.795214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:43.810420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 557/1101] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:43.925373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3d11-8df8-4495-8de2-2498555adc9b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:43.966818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:43.967456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:43.967727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:43.968022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:43.968797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:43.969104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:43.970650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:43.991777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 545/1102] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 6 11:37:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:43.998531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:44.183903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf4b24-ea7b-4491-aff6-0a37a89a78f8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:44.223916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:44.242714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:44.243553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:44.243930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:44.244347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:44.300964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:44.337876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:44.351052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:44.353053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:44.385777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5818-59e7-4dbf-9a04-4999dbeb0264 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:44.452497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:44.453617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:44.454170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:44.454645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:44.456741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:44.460020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:44.463326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 546/1103] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:44.470554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:44.479857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 558/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:37:44.485974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:44.606510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:44.619073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a94e-933f-4dba-8c81-b917c49c042e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:44.628535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a1b1c-54af-494a-9a0a-4522b20d7396 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:44.649983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:44.666819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:44.667474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:44.667746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:44.668016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:44.680595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:44.680870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:44.681446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:44.681856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:44.682099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:44.682302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:44.683038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:44.684666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:37:44.702042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:44.704289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:37:44.705339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:37:44.705339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:37:44.705339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:37:44.705339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 559/1105] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:37:44.711086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:44.729706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 547/1106] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:37:44 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:37:44.905782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5f4c-4bff-409d-b371-1ce849e223ab None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:44.949118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:44.949793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:44.950251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:44.950530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:44.951208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:44.951517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:44.952940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:44.976392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 560/1107] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:44.982691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:45.172005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c2c94-00c6-49ea-8b6b-0e5ad4120e25 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:45.230283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:45.231387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:45.231387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:45.231671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:45.240460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:45.242162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:37:45.259974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:37:45.271181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:37:45.273197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 548/1108] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:37:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:37:45.274219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:45.276697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:45.392551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bd1f9-25cf-4dd8-8b94-c50d08038083 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:45.451467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:45.451979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:45.452261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:45.452530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:45.453235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:45.453535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:45.455028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a02f52-af52-4d13-a872-247bc9e4a69e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:45.469132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 561/1109] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:45.476538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:45.515744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:45.602482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4855-dda7-45de-bd74-b21155b4f7f4 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:45.643666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:45.644281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:45.644561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:45.645110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:45.646139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] RBAC: Adding query filter params (role.id=86c64cac25b34439972a071e46a61bca, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=9260fd9f9c524d20a9025595a53ade4e) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:45.646303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:45.647670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575b767-9536-41fe-860e-7a71fff2fc77 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:45.654743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 549/1110] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:45 2022] GET /identity/v3/role_assignments?user.id=9260fd9f9c524d20a9025595a53ade4e&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=86c64cac25b34439972a071e46a61bca => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:46.869502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dd788-971d-4053-8924-325d2ed61519 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:46.870190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dd788-971d-4053-8924-325d2ed61519 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:46.870468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dd788-971d-4053-8924-325d2ed61519 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:46.872857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 562/1111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:46.878993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:46.880227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:46.880384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:46.908426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:46.909638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:46.985542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 550/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:47.276745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dd788-971d-4053-8924-325d2ed61519 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:47.355342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:47.403692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dd788-971d-4053-8924-325d2ed61519 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:47.443188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:47.443826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:47.444099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:47.444365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] PATH_INFO: `/v3/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:47.445942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] RBAC: Authorizing `identity:get_role(role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:47.447374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:47.448756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 563/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:47 2022] GET /identity/v3/roles/acebbc3107e1448599b7b2bd0096b755 => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:47.454823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:47.485051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:47.572975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc86c73-fb67-479a-8012-bc07e06c4598 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:47.624528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:47.625464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:47.625922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:47.626333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] PATH_INFO: `/v3/users/ee507e3f496b4e8b83e768b07a68d6b7` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:47.644093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] RBAC: Authorizing `identity:get_user(user_id=ee507e3f496b4e8b83e768b07a68d6b7)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:47.645977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:47.647846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 551/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:47 2022] GET /identity/v3/users/ee507e3f496b4e8b83e768b07a68d6b7 => generated 286 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:47.654341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:47.772695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41bf0-9f60-4da6-91d1-2a0372700a1e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:47.807498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:47.813509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:47.814168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:47.814440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:47.814710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:47.920809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:47.922544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:48.091374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 564/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:37:48.097781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:48.198732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:48.218798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ea0a-45d4-4cd8-b4cb-b54aca3c8dbc None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:48.265589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:48.266365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:48.266630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:48.266846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:48.268077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:48.270227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965edc-822d-49b8-93f3-38266d8f5b7c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:48.272082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 552/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:48 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:48.278962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:48.316153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:48.401717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c6ae-ddd6-4a04-bc41-df6a15161f9d None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:48.440909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:48.441922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:48.442259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:48.442259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:48.443200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06, user.id=ee507e3f496b4e8b83e768b07a68d6b7) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:48.443508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:48.444997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3f76c-863d-4f1c-ab16-24dc26c2ecb4 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:48.451840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 565/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:48 2022] GET /identity/v3/role_assignments?user.id=ee507e3f496b4e8b83e768b07a68d6b7&scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:50.444949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ece5-d2d8-4ae1-8f64-e47b624f4939 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:50.445541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ece5-d2d8-4ae1-8f64-e47b624f4939 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:50.445541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ece5-d2d8-4ae1-8f64-e47b624f4939 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:50.447045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 553/1118] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:50.452799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:50.453352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:50.453352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:50.475570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:50.477095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:50.563264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 566/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:50.846718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ece5-d2d8-4ae1-8f64-e47b624f4939 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:50.964968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ece5-d2d8-4ae1-8f64-e47b624f4939 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:51.005603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:51.006326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:51.006599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:51.006865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] PATH_INFO: `/v3/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:51.008330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] RBAC: Authorizing `identity:get_role(role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:51.009783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:51.011458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 554/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:50 2022] GET /identity/v3/roles/acebbc3107e1448599b7b2bd0096b755 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:51.017602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:51.048129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:51.130212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cee81e-5947-40c9-b8b0-553570dac9fb None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:51.170581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:51.171235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:51.171508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:51.171770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] PATH_INFO: `/v3/users/ee507e3f496b4e8b83e768b07a68d6b7` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:51.172996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] RBAC: Authorizing `identity:get_user(user_id=ee507e3f496b4e8b83e768b07a68d6b7)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:51.174497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:51.176681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 567/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:51 2022] GET /identity/v3/users/ee507e3f496b4e8b83e768b07a68d6b7 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:51.183016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:51.209568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:51.305944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d11637-206f-44f6-99e2-daf9640adb08 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:51.345608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:51.346295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:51.346573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:51.346846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:51.348189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:51.350078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38017247-34f6-460a-b48c-a0986798f9b9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:51.351823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 555/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:51 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:51.360330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:51.464173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:51.471531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae869e-0fc2-473c-abcc-af325cd4b222 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:51.509116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:51.509832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:51.510113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:51.510386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06/users/ee507e3f496b4e8b83e768b07a68d6b7/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:51.512848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] RBAC: Authorizing `identity:create_grant(project_id=e4dc8e171e884e31bc7b1546ee7bfd06, user_id=ee507e3f496b4e8b83e768b07a68d6b7, role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:51.514336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f3f8b-d825-41ad-b065-5cb5590db61e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:51.528463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 568/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:37:51 2022] PUT /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06/users/ee507e3f496b4e8b83e768b07a68d6b7/roles/acebbc3107e1448599b7b2bd0096b755 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:37:53.586113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa7d0-70be-492c-b040-9fc690bea5c9 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:53.586685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa7d0-70be-492c-b040-9fc690bea5c9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:53.587009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa7d0-70be-492c-b040-9fc690bea5c9 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:53.589675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 556/1124] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:53.595779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:53.596314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:53.596485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:53.617935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:53.619443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:53.700870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 569/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:54.000780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa7d0-70be-492c-b040-9fc690bea5c9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:54.108872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:54.124037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa7d0-70be-492c-b040-9fc690bea5c9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:54.162239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:54.162889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:54.163171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:54.163475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] PATH_INFO: `/v3/roles/acebbc3107e1448599b7b2bd0096b755` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:54.165087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] RBAC: Authorizing `identity:get_role(role_id=acebbc3107e1448599b7b2bd0096b755)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:54.166548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:54.167942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 557/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:53 2022] GET /identity/v3/roles/acebbc3107e1448599b7b2bd0096b755 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:54.174154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:54.273039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:54.291184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b63d0c-5fbc-49a6-af56-8007a90da57b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:54.333028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:54.333716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:54.334048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:54.334366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] PATH_INFO: `/v3/users/ee507e3f496b4e8b83e768b07a68d6b7` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:54.335677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] RBAC: Authorizing `identity:get_user(user_id=ee507e3f496b4e8b83e768b07a68d6b7)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:54.337262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:54.339143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 570/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:37:54 2022] GET /identity/v3/users/ee507e3f496b4e8b83e768b07a68d6b7 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:54.345362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:54.461092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d564f50-5b9d-44a6-a882-7f3fd3a8969b None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:54.499387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:54.500008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:54.500286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:54.500555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:54.529961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:54.583400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:54.584876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:54.695629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 558/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:37:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:37:54.704783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:54.834142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405f6-7682-453c-b5d7-2a07e7db35d2 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:54.876463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:54.877133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:54.877407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:54.877710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] PATH_INFO: `/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:54.879548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] RBAC: Authorizing `identity:get_project(project_id=e4dc8e171e884e31bc7b1546ee7bfd06)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:54.881146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bfde3-3801-437d-b60d-0540ffc5502e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:54.882959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 571/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:37:54 2022] GET /identity/v3/projects/e4dc8e171e884e31bc7b1546ee7bfd06 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:54.890492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:54.963330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:55.022850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc5da4-d737-4b5d-9cb0-e77cf29abb21 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:55.065954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:55.067820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:55.068274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:55.068758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:55.070194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] RBAC: Adding query filter params (role.id=acebbc3107e1448599b7b2bd0096b755, scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06, user.id=ee507e3f496b4e8b83e768b07a68d6b7) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:37:55.070754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:55.073459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235dd16-d85b-4330-be28-1e3c70c3e746 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:55.085171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 559/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:37:54 2022] GET /identity/v3/role_assignments?user.id=ee507e3f496b4e8b83e768b07a68d6b7&scope.project.id=e4dc8e171e884e31bc7b1546ee7bfd06&role.id=acebbc3107e1448599b7b2bd0096b755 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:37:57.213108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77882df-1a0a-4597-8cce-bbe68604d149 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:57.213729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77882df-1a0a-4597-8cce-bbe68604d149 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:57.213729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77882df-1a0a-4597-8cce-bbe68604d149 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:57.215023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 572/1131] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:37:57.220983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:57.221565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:57.221565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:57.241670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:37:57.242901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:37:57.330446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 560/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:37:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:37:57.606371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77882df-1a0a-4597-8cce-bbe68604d149 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:57.630915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:57.728397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77882df-1a0a-4597-8cce-bbe68604d149 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:57.774532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:57.775181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:57.775456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:57.775724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:57.776919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:57.778408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60bcb-cf7d-4022-8146-19aec6790106 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:57.780071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 573/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:37:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:37:57.787015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:37:57.843276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:37:57.914393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a398049-fda8-44e5-8f83-a09a18c2d17d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:37:57.969862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:37:57.970847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:37:57.971228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:37:57.971688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:37:57.973731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:37:57.976150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba9d99-1343-45b1-ac06-ff505e516329 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:37:58.016572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 561/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:37:57 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:38:00.077132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f09e2-0482-48a2-b444-21f210018ad5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:00.077764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f09e2-0482-48a2-b444-21f210018ad5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:00.077764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f09e2-0482-48a2-b444-21f210018ad5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:00.079076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 574/1135] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:00.086496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:00.087073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:00.087073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:00.106184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:00.107404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:00.185749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 562/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:38:00.565064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f09e2-0482-48a2-b444-21f210018ad5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:00.747940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:00.777306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f09e2-0482-48a2-b444-21f210018ad5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:00.833106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:00.833965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:00.834245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:00.834515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] PATH_INFO: `/v3/domains/default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:00.835857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:00.837352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405852c-e2a8-4245-aa70-1aedd10749be None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:00.839134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 575/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:38:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:00.846516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:00.890437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:01.020825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04187ead-7955-4e2a-aca0-3f02a5198c25 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:01.086957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:01.087885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:01.088373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:01.088852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:01.090153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:01.093009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca4f88-7046-41fb-a2b7-a7ffa2f494fa None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:01.125904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 563/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:38:00 2022] POST /identity/v3/users => generated 286 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:38:03.245927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074356d-df75-4d18-af77-4ab9e83340b4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:03.246788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074356d-df75-4d18-af77-4ab9e83340b4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:03.247028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074356d-df75-4d18-af77-4ab9e83340b4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:03.248624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 576/1139] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:03.256322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:03.256876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:03.256876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:03.277641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:03.278934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:03.359063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 564/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:03.652220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074356d-df75-4d18-af77-4ab9e83340b4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:03.775580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074356d-df75-4d18-af77-4ab9e83340b4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:03.815314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:03.815959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:03.816895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:03.817632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:03.829387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:03.835645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:03.837674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:03.839472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 577/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:38:03.843958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:03.879427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:04.022345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a2dae-1b16-45e4-ab72-b85174428f9b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:04.063030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:04.063657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:04.063937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:04.064208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:04.064896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:04.065202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:04.066756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:04.076185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 565/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:04.085074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:04.105168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:04.197619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372e5b0-954d-45ff-abe4-a4c6407c8f26 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:04.252558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:04.253234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:04.253514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:04.253844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] PATH_INFO: `/v3/users/5f4e119316b645d1973c2518a9742e64` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:04.271976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] RBAC: Authorizing `identity:get_user(user_id=5f4e119316b645d1973c2518a9742e64)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:04.273532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:04.275285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 578/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:04 2022] GET /identity/v3/users/5f4e119316b645d1973c2518a9742e64 => generated 286 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:04.283286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:04.386370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:04.407916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfa8c9-fed0-4530-8f6c-79cdda504734 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:04.447374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:04.448159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:04.448437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:04.448708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:04.531285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:04.532781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:04.633975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:04.658243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 566/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:38:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:38:04.665287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:04.756328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:04.786841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e4d93-c789-455d-9bf5-70b474b29e6e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:04.827225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:04.827931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:04.828205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:04.828475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] PATH_INFO: `/v3/projects/ad60beadda7d4bf09edf0d1750319fc4` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:04.829723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] RBAC: Authorizing `identity:get_project(project_id=ad60beadda7d4bf09edf0d1750319fc4)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:04.831189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974cd1-0287-44a0-9961-1f7eb17b7948 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:04.833369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 579/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:04 2022] GET /identity/v3/projects/ad60beadda7d4bf09edf0d1750319fc4 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:38:04.840441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:04.992324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55eb9b-8a75-49e0-9578-d38184db4fb5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:05.042582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:05.043122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:05.043397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:05.043665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:05.044419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ad60beadda7d4bf09edf0d1750319fc4, user.id=5f4e119316b645d1973c2518a9742e64) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:05.044725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:05.046270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44adb4-2561-4692-8a3d-5d50536b72fb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:05.053254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 567/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:38:04 2022] GET /identity/v3/role_assignments?user.id=5f4e119316b645d1973c2518a9742e64&scope.project.id=ad60beadda7d4bf09edf0d1750319fc4&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:38:07.072628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6253ae-b331-4cc7-91ec-4f3730e201ba None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:07.073280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6253ae-b331-4cc7-91ec-4f3730e201ba None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:07.073280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6253ae-b331-4cc7-91ec-4f3730e201ba None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:07.074755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 580/1147] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:38:07.080886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:07.081435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:07.081612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:07.102223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:07.103521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:07.151791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:07.184505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 568/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:38:07.475530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6253ae-b331-4cc7-91ec-4f3730e201ba None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:07.587860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6253ae-b331-4cc7-91ec-4f3730e201ba None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:07.627842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:07.628609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:07.628888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:07.629158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:07.640948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:07.646547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:07.648466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:07.649924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 581/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:38:07.654046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:07.772932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcd4b3-93cb-46c8-bcc4-093681794bc7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:07.813530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:07.814245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:07.814569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:07.814886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:07.815612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:07.816191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:07.818308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:07.828046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 569/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:07.834989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:07.838745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:07.952650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca729ea-9c4d-49df-b62b-08e2ac70f2a1 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:07.991341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:07.991999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:07.992274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:07.992542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] PATH_INFO: `/v3/users/5f4e119316b645d1973c2518a9742e64` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:07.993756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] RBAC: Authorizing `identity:get_user(user_id=5f4e119316b645d1973c2518a9742e64)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:07.995255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:07.996977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 582/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:07 2022] GET /identity/v3/users/5f4e119316b645d1973c2518a9742e64 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:08.003244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:08.047393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:08.125614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd96e7-56e1-4991-b9c8-74c1679a6b72 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:08.164357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:08.165004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:08.165275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:08.165541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] PATH_INFO: `/v3/projects/ad60beadda7d4bf09edf0d1750319fc4` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:08.166965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] RBAC: Authorizing `identity:get_project(project_id=ad60beadda7d4bf09edf0d1750319fc4)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:08.168448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2960397-86c0-4cec-aaae-ae06eb6ba53c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:08.170219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 570/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:08 2022] GET /identity/v3/projects/ad60beadda7d4bf09edf0d1750319fc4 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:08.180593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:08.288510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:08.326397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390eed-aa20-4b90-8478-b3b6084a3a1c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:08.365751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:08.366473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:08.366746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:08.367014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] PATH_INFO: `/v3/projects/ad60beadda7d4bf09edf0d1750319fc4/users/5f4e119316b645d1973c2518a9742e64/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:08.369178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad60beadda7d4bf09edf0d1750319fc4, user_id=5f4e119316b645d1973c2518a9742e64, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:08.370673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893dbd6-52db-44ed-af65-f95a8281a295 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:08.387027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 583/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:38:08 2022] PUT /identity/v3/projects/ad60beadda7d4bf09edf0d1750319fc4/users/5f4e119316b645d1973c2518a9742e64/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:38:10.414175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf71c3-b11e-467a-b591-d0c74b8962ae None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:10.414958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf71c3-b11e-467a-b591-d0c74b8962ae None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:10.414958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf71c3-b11e-467a-b591-d0c74b8962ae None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:10.416180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 571/1154] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:38:10.422609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:10.423203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:10.423203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:10.443707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:10.444925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:10.526203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 584/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:10.901382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf71c3-b11e-467a-b591-d0c74b8962ae None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:11.033406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf71c3-b11e-467a-b591-d0c74b8962ae None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:11.036911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:11.085754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:11.086807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:11.087288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:11.087761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:11.107040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:11.116951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:11.118248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:38:11.119129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 572/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:38:11.125910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:11.195738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:11.244159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea0746-89c9-4199-a0b6-969ef5023933 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:11.287395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:11.288011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:11.288284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:11.288550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:11.289228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:11.289527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:11.290949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:11.303032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 585/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:11.309180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:11.427463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343dcad-3166-4382-bb80-243629316f0a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:11.469121 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:11.469712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:11.469987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:11.470249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] PATH_INFO: `/v3/users/5f4e119316b645d1973c2518a9742e64` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:11.471505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] RBAC: Authorizing `identity:get_user(user_id=5f4e119316b645d1973c2518a9742e64)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:11.473372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:11.475141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 573/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:11 2022] GET /identity/v3/users/5f4e119316b645d1973c2518a9742e64 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:11.481443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:11.607679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1657b4-7353-4408-bf1b-86e991980428 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:11.635820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:11.650543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:11.651175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:11.651451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:11.651723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:11.729859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:11.731263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:11.848722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 586/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:38:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:38:11.854729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:11.890917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:11.968216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea09da-6665-40cf-b666-445e9cd08d58 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:12.008322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:12.008882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:12.009160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:12.009431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] PATH_INFO: `/v3/projects/ad60beadda7d4bf09edf0d1750319fc4` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:12.010670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] RBAC: Authorizing `identity:get_project(project_id=ad60beadda7d4bf09edf0d1750319fc4)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:12.012402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c045e7d-337d-4326-8e23-f72987d53982 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:12.014141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 574/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:11 2022] GET /identity/v3/projects/ad60beadda7d4bf09edf0d1750319fc4 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:12.024888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:12.052954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:12.136430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88687-83d0-47c9-ba71-605c9113c8f6 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:12.177241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:12.177893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:12.178229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:12.178506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:12.179241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=ad60beadda7d4bf09edf0d1750319fc4, user.id=5f4e119316b645d1973c2518a9742e64) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:12.179542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:12.181165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c368a-ba4f-49c1-b51c-2c8543983c31 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:12.188138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 587/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:38:12 2022] GET /identity/v3/role_assignments?user.id=5f4e119316b645d1973c2518a9742e64&scope.project.id=ad60beadda7d4bf09edf0d1750319fc4&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:14.325078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa59d-5d5f-49ef-9423-0632a72cf082 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:14.325768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa59d-5d5f-49ef-9423-0632a72cf082 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:14.325768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa59d-5d5f-49ef-9423-0632a72cf082 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:14.327036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 575/1162] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:38:14.336682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:14.337284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:14.337284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:14.356467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:14.357717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:14.434703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 588/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:14.709573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa59d-5d5f-49ef-9423-0632a72cf082 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:14.826652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:14.832219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa59d-5d5f-49ef-9423-0632a72cf082 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:14.871781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:14.872411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:14.872700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:14.872969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:14.946587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:14.948049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:15.050481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:15.055015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 576/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:38:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:38:15.060832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:15.083356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:15.185904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2448d5-59b4-4072-b1c4-6790732d926a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:15.226547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:15.227198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:15.227474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:15.227741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:15.235691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:15.237175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-da204966-3362-466f-9c62-0efaa5c9020d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:15.251690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:15.253256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:15.254221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 589/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:38:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:38:15.263530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:15.387330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055b7b-6ad1-4c21-9363-bc9136349ae8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:15.453290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:15.453945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:15.454220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:15.454487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:15.455147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:15.455567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:15.457060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462a4be-0c2c-4cd1-a71d-6b2fb1b65200 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:15.465813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 577/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 6 11:38:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:17.511809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6ad9-2b42-4203-a12b-a3e50a601bdf None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:17.512432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6ad9-2b42-4203-a12b-a3e50a601bdf None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:17.512432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6ad9-2b42-4203-a12b-a3e50a601bdf None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:17.513606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 590/1167] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:17.520103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01af7-ad38-408e-b20a-ceb38dd6b23b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:17.520635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01af7-ad38-408e-b20a-ceb38dd6b23b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:17.520861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01af7-ad38-408e-b20a-ceb38dd6b23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:17.541850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4ed01af7-ad38-408e-b20a-ceb38dd6b23b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:17.543072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4ed01af7-ad38-408e-b20a-ceb38dd6b23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:17.615452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:17.625790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 578/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:17.902900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6ad9-2b42-4203-a12b-a3e50a601bdf None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:17.995224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:18.023480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6ad9-2b42-4203-a12b-a3e50a601bdf None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:18.063940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:18.064563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:18.064840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:18.065107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:18.066808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:18.068509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5c472-83e9-4bd8-a96d-3c23a5d23563 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:18.087663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 591/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Tue Sep 6 11:38:17 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:38:20.205767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58a6-75a4-4688-a2b5-91401a25fdc8 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:20.206345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58a6-75a4-4688-a2b5-91401a25fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:20.206570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58a6-75a4-4688-a2b5-91401a25fdc8 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:20.209579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 579/1170] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:20.215641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629498-e1a6-4349-a21b-40069d332999 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:20.216164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629498-e1a6-4349-a21b-40069d332999 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:20.216326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629498-e1a6-4349-a21b-40069d332999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:20.244099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-37629498-e1a6-4349-a21b-40069d332999 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:20.245608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-37629498-e1a6-4349-a21b-40069d332999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:20.329333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 592/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:20.614839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a58a6-75a4-4688-a2b5-91401a25fdc8 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:20.740940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a58a6-75a4-4688-a2b5-91401a25fdc8 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:20.786615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:20.787277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:20.787557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:20.787832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] PATH_INFO: `/v3/domains/d646196f0b9e4fc08681f996b37aa28f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:20.805499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d646196f0b9e4fc08681f996b37aa28f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:20.807650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33b913-f98b-45fb-853c-9233975b0835 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:20.809460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 580/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 6 11:38:20 2022] GET /identity/v3/domains/d646196f0b9e4fc08681f996b37aa28f => generated 267 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:20.815628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37629498-e1a6-4349-a21b-40069d332999 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:20.909246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:20.975757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37629498-e1a6-4349-a21b-40069d332999 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:21.041987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:21.042624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:21.042898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:21.043162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:21.043836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:21.045334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9cfd9-bcd3-4a23-ae8c-45796c28b23e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:21.078514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 593/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:38:20 2022] POST /identity/v3/projects => generated 361 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:38:23.174852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866670c-ad0c-480b-b12f-b88b21d15a04 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:23.175488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866670c-ad0c-480b-b12f-b88b21d15a04 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:23.176710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866670c-ad0c-480b-b12f-b88b21d15a04 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:23.178180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 581/1174] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:38:23.184248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:23.184827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:23.184827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:23.204303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:23.205742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:23.281428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 594/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:38:23.564477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866670c-ad0c-480b-b12f-b88b21d15a04 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:23.629549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:23.682301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866670c-ad0c-480b-b12f-b88b21d15a04 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:23.720540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:23.721180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:23.721455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:23.721751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] PATH_INFO: `/v3/domains/d646196f0b9e4fc08681f996b37aa28f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:23.723652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=d646196f0b9e4fc08681f996b37aa28f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:23.725235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c764d9-0e62-4c0b-b9ce-13da99cc12bc None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:23.727009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 582/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 6 11:38:23 2022] GET /identity/v3/domains/d646196f0b9e4fc08681f996b37aa28f => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:23.737276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:23.796886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:23.860505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3061-d5fb-4881-9770-39ff56d862c2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:23.903858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:23.904609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:23.904978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:23.905347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:23.906194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:23.907805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b22d4-3944-40b7-8f0d-7b357d14b19b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:23.934682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 595/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:38:23 2022] POST /identity/v3/users => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:38:26.121377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea730-0a00-4857-bfbb-ccb30c8ff4b7 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:26.122008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea730-0a00-4857-bfbb-ccb30c8ff4b7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:26.122008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea730-0a00-4857-bfbb-ccb30c8ff4b7 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:26.123329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 583/1178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:26.130800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:26.131353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:26.131750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:26.163011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:26.165019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:26.255859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 596/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:26.532939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea730-0a00-4857-bfbb-ccb30c8ff4b7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:26.661848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea730-0a00-4857-bfbb-ccb30c8ff4b7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:26.689378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:26.705385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:26.706074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:26.706685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:26.707340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:26.719858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:26.726473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:26.727848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:26.728947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:38:26.732920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 584/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:38:26.734629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:26.792945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:26.871440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629acbe-e629-4ea5-a71e-e61113676cc2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:26.911122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:26.911928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:26.912212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:26.912480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:26.913188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:26.913496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:26.914948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:26.924712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 597/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:26.931561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:27.050686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a78-167b-4fea-b688-da34b48a7b7a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:27.094743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:27.095511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:27.095834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:27.096148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] PATH_INFO: `/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:27.113576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] RBAC: Authorizing `identity:get_user(user_id=5c6270fffa5b4d44a359ac7d3df20dc4)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:27.115371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:27.117210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 585/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:26 2022] GET /identity/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:27.124106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:27.247388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:27.256474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16800e7f-e439-4d9b-bb1e-a7bc9cd8e701 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:27.295698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:27.296462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:27.296736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:27.297002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:27.376838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:27.378422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:27.490149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:27.498655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 598/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:38:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:38:27.505729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:27.554715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:27.625304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ea66f-14c3-417d-b5fa-f1bf1f7a5787 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:27.667152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:27.667798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:27.668072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:27.668337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] PATH_INFO: `/v3/projects/dd3091dddb2e433092c53f5e0bc974ea` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:27.669511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] RBAC: Authorizing `identity:get_project(project_id=dd3091dddb2e433092c53f5e0bc974ea)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:27.671018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ef91-19b8-43cb-ab5f-1cf0afb8c829 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:27.672748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 586/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:27 2022] GET /identity/v3/projects/dd3091dddb2e433092c53f5e0bc974ea => generated 361 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:27.679986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:27.799585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c58933-c94f-4450-aebb-e234677b9972 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:27.840641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:27.841264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:27.841543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:27.841843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:27.842590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=dd3091dddb2e433092c53f5e0bc974ea, user.id=5c6270fffa5b4d44a359ac7d3df20dc4) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:27.842895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:27.844363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba221a-d823-40d7-ac29-f3731576a4f1 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:27.851212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 599/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:38:27 2022] GET /identity/v3/role_assignments?user.id=5c6270fffa5b4d44a359ac7d3df20dc4&scope.project.id=dd3091dddb2e433092c53f5e0bc974ea&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:29.897756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc310-824b-43bf-9663-e175a347634d None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:29.898471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc310-824b-43bf-9663-e175a347634d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:29.898471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc310-824b-43bf-9663-e175a347634d None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:29.899763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 587/1186] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:29.909183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:29.909790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:29.910166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:29.932032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:29.933275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:29.996712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:30.010254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 600/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:30.289049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc310-824b-43bf-9663-e175a347634d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:30.410015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc310-824b-43bf-9663-e175a347634d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:30.421932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:30.452348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:30.453158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:30.453555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:30.453971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:30.466907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:30.472610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:30.474896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:38:30.476647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 588/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:38:30.484554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:30.683002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79221a32-df5f-4ac4-ad21-67e0a4ecf832 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:30.756508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:30.757809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:30.758439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:30.758907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:30.760027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:30.760470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:30.763169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:30.780076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 601/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:30.788219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:30.910522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9d6f9-2a07-40c6-9e6d-6172c82eaa76 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:30.949784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:30.950443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:30.950762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:30.951076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] PATH_INFO: `/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:30.952334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] RBAC: Authorizing `identity:get_user(user_id=5c6270fffa5b4d44a359ac7d3df20dc4)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:30.953917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:30.955659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 589/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:30 2022] GET /identity/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:30.962496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:31.025800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:31.093142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e3ca3-c53e-49e4-8eaf-c080f3964307 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:31.136053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:31.137021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:31.137368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:31.137711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] PATH_INFO: `/v3/projects/dd3091dddb2e433092c53f5e0bc974ea` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:31.139161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] RBAC: Authorizing `identity:get_project(project_id=dd3091dddb2e433092c53f5e0bc974ea)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:31.140684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f69c0-8515-41f8-8323-335a54ffaef8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:31.142674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 602/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:30 2022] GET /identity/v3/projects/dd3091dddb2e433092c53f5e0bc974ea => generated 361 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:38:31.149948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:31.196788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:31.277844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbea365-2116-46cd-8ccd-b872859e2ccd None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:31.320420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:31.321099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:31.321378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:31.321677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] PATH_INFO: `/v3/projects/dd3091dddb2e433092c53f5e0bc974ea/users/5c6270fffa5b4d44a359ac7d3df20dc4/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:31.323789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd3091dddb2e433092c53f5e0bc974ea, user_id=5c6270fffa5b4d44a359ac7d3df20dc4, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:31.325487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e384d-1493-456f-b799-3dd48bbeb2f4 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:31.341887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 590/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:38:31 2022] PUT /identity/v3/projects/dd3091dddb2e433092c53f5e0bc974ea/users/5c6270fffa5b4d44a359ac7d3df20dc4/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:38:33.382964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c38e2-2cb8-48c3-8da7-70a0f6db5752 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:33.383563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c38e2-2cb8-48c3-8da7-70a0f6db5752 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:33.383563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c38e2-2cb8-48c3-8da7-70a0f6db5752 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:33.384898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 603/1193] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:38:33.391151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:33.391712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:33.391712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:33.413400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:38:33.414671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:38:33.498752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 591/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:38:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:38:33.773235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c38e2-2cb8-48c3-8da7-70a0f6db5752 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:33.895141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c38e2-2cb8-48c3-8da7-70a0f6db5752 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:33.918802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:33.936068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:33.936807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:33.937131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:33.937448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:33.950851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:38:33.958260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:38:33.960047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:38:33.961674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 604/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:38:33.966022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:33.997296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:34.084317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decfa72-b9b8-4d8b-a66f-5e238fb8fac0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:34.130895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:34.131523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:34.131797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:34.132064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:34.132752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:34.133049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:34.134527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:34.144191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 592/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:38:34.150668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:34.272162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af2f3-07dc-41ff-89fd-99fc29eebba4 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:34.311846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:34.312496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:34.312769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:34.313037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] PATH_INFO: `/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:34.314289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] RBAC: Authorizing `identity:get_user(user_id=5c6270fffa5b4d44a359ac7d3df20dc4)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:34.316295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:34.318013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 605/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 6 11:38:34 2022] GET /identity/v3/users/5c6270fffa5b4d44a359ac7d3df20dc4 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:38:34.324139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:34.427758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:34.443166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f417f0-9715-40c5-b72a-255a6df9cca5 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:34.481511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:34.482343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:34.482667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:34.482984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:34.562552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:34.564076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:34.686498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:34.703490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 593/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:38:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 11:38:34.710564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:34.771624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:34.832390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcbd1-118f-4204-9e1c-6306eb60365a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:34.874875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:34.875534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:34.875833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:34.876147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] PATH_INFO: `/v3/projects/dd3091dddb2e433092c53f5e0bc974ea` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:34.877543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] RBAC: Authorizing `identity:get_project(project_id=dd3091dddb2e433092c53f5e0bc974ea)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:34.879711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5594f-d45e-467c-860c-7f320b742d0f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:34.881533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 606/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 6 11:38:34 2022] GET /identity/v3/projects/dd3091dddb2e433092c53f5e0bc974ea => generated 361 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:34.891240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:38:35.005247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ddce9-d24b-4c9c-b7d4-3f6b27b1e620 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:38:35.044537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:38:35.045156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:38:35.045428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:38:35.045718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:38:35.046460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=dd3091dddb2e433092c53f5e0bc974ea, user.id=5c6270fffa5b4d44a359ac7d3df20dc4) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:38:35.046764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:38:35.048248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c044-2c4a-43e6-b5cf-15ea0f4352a3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:38:35.055637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 594/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:38:34 2022] GET /identity/v3/role_assignments?user.id=5c6270fffa5b4d44a359ac7d3df20dc4&scope.project.id=dd3091dddb2e433092c53f5e0bc974ea&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:38:44.870139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=42 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:38:45.052437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=74 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:37.820041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3963-9f3b-499a-90d4-8f274f500b6d None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:37.820688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3963-9f3b-499a-90d4-8f274f500b6d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:37.821035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3963-9f3b-499a-90d4-8f274f500b6d None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:37.823499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 607/1201] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:39:37.830493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:37.831286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:37.831286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:37.854801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:39:37.856205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:39:38.001578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 595/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:39:38.323867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e3963-9f3b-499a-90d4-8f274f500b6d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:38.536014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e3963-9f3b-499a-90d4-8f274f500b6d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:38.604222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:38.604933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:38.605215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:38.605492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:38.619387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:38.620881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:38.656931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:38.658396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:38.659402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:38.659402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:38.659402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:38.660532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 608/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:38.666239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:38.871960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180e423-11c7-49e9-829c-48c089fe446a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:38.917858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:38.932671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:38.933345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:38.933725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:38.934010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:38.934709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:38.935108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:38.936606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc252ca-0de2-48c9-ac0c-13c78ca84772 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:38.952472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 596/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:38.960512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:39.079823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:39.180100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f690-3640-4c5c-8cd5-1640185ccb5f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:39.265317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:39.266095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:39.266407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:39.266871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:39.267688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:39.269231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984de0e-5a45-436f-8c8d-e0b4d8c8c026 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:39.296873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 609/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 6 11:39:38 2022] POST /identity/v3/users => generated 251 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:39:42.107489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650666-603e-4469-bc3f-0d1b7927f139 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:42.108464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650666-603e-4469-bc3f-0d1b7927f139 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:42.108464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650666-603e-4469-bc3f-0d1b7927f139 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:42.109591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 597/1206] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:39:42.116301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:42.117141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:42.117141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:42.139857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:39:42.141062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:39:42.265754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 610/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:39:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:39:42.577788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650666-603e-4469-bc3f-0d1b7927f139 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:42.781856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650666-603e-4469-bc3f-0d1b7927f139 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:42.858037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:42.858833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:42.859245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:42.859677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:42.881206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:39:42.887690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:42.890418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:42.892138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:42.895154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 598/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:39:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:42.896658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:42.986433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:43.116672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15abde-3919-4b17-a64b-f9a1bc4afc32 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:43.177894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:43.178992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:43.179523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:43.179824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:43.180564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:43.180983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:43.182709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:43.197355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 611/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:39:43.206753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:43.296384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:43.374532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641992eb-c8b5-4242-991d-f5c27439e30a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:43.428241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:43.429096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:43.429275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:43.429414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:43.444354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:43.445896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:43.463304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:43.466334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:43.468417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:43.468417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:43.468417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:43.468417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 599/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:43.476321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:43.713769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:43.721341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540760c-605b-4c28-b14b-272e77bc8c17 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:43.789032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:43.789812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:43.790267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:43.790267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:43.790939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:43.791243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:43.792721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:43.808446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 612/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:43.818545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:44.011767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a0274-8953-439e-8221-fc43bfcd2730 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:44.014892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:44.069866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:44.070955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:44.071369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:44.071831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:44.085153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:44.088039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:39:44.104936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:44.106798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:44.108737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 600/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:44.115473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:44.348268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21281e-c710-44f9-875c-59900362f2b9 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:44.432079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:44.433589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:44.433919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:44.434386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:44.435849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:44.436504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:44.439962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:44.473081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 613/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:39:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:44.481399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:44.700627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb87-b194-4eb1-854f-e16b317441eb None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:44.745081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:44.746741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:44.747067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:44.747351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:44.791015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:44.856822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:44.860719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:45.072424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 601/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:39:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:39:45.080402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:45.117024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:45.278358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2579dee-3cec-4912-906c-32be4cca2817 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:45.349486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:45.350504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:45.350989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:45.351473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:45.380422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:45.383299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:45.414703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:45.416408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:45.418997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:45.418997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:45.418997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:45.418997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 614/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:45.426941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:45.558414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:45.638971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228aaff9-68f5-4a4e-8df7-8af67e774ab9 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:45.694907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:45.695687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:45.696057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:45.696512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:45.697550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:45.698134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:45.700747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:45.725170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 602/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:45.734169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:45.855920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:45.913397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418329c9-0456-4977-afad-b8abc38f88db None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:45.990069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:45.991361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:45.991873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:45.992335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:46.005492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:46.008392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:39:46.028152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:46.030527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:46.033329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:46.035456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 615/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:46.041026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:46.260128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153abe9-88ac-4201-bc38-80b1b7d75444 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:46.292794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:46.308340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:46.308979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:46.309264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:46.309512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:46.310288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:46.310595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:46.312130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86596140-c801-40b4-b5bf-d47bf4022eed None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:46.338388 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 603/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:39:46.344643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:46.543937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f890-0389-4405-be33-12f912f57903 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:46.619704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:46.630826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:46.631570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:46.631868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:46.632313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:46.633368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=8fd64b822c934dada9d93d8179e374ec) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:46.633711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:46.635802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49883432-f6c0-42e7-bbc5-2397840ca336 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:46.645016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 616/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:39:46 2022] GET /identity/v3/role_assignments?user.id=8fd64b822c934dada9d93d8179e374ec&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:48.852629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c8231-090e-4f8a-b0ce-919cd34b15d4 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:48.853487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c8231-090e-4f8a-b0ce-919cd34b15d4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:48.853487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c8231-090e-4f8a-b0ce-919cd34b15d4 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:48.854548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 604/1220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:39:48.864131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:48.864695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:48.865035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:48.893276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:39:48.894806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:39:49.019639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 617/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:39:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:39:49.321923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c8231-090e-4f8a-b0ce-919cd34b15d4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:49.482680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c8231-090e-4f8a-b0ce-919cd34b15d4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:49.539632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:49.540608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:49.541073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:49.541539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:49.562654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:39:49.573157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:49.575386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:49.577423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 605/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:39:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:49.584832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:49.775292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05f8f-1526-4b55-9d22-546f36421505 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:49.810336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:49.830108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:49.831019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:49.831394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:49.831812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:49.832658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] RBAC: Adding query filter params (name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:49.833175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:49.834925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:49.848740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 618/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:39:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:49.859239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:49.931715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:50.020123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec89dc4-97be-4450-a1d7-7340b9d0a10d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:50.073029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:50.073721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:50.074053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:50.074392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:50.093144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:50.095506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:50.117053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:50.119321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:50.120349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:50.120349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:50.120349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:50.120349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 606/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:50.126652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:50.288049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1546245-4d6c-4120-bcf3-43fbe5e31c4f None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:50.343131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:50.343956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:50.345133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:50.345606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:50.346555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:50.347212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:50.349365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:50.370241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 619/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:39:50.376981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:50.538640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:50.549353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd7f4-1bb2-45f1-8095-1b9c8e03e0fe None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:50.613504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:50.613970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:50.614183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:50.614437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:50.626227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:50.628507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:39:50.644938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:50.646845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:50.648149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 607/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:50.653905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:50.713887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:50.815212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaee1d-f9bd-4c37-9340-1a0a2a8554c0 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:50.867075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:50.867751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:50.868114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:50.868482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:50.869343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:50.869798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:50.871438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:50.895491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 620/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:39:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:39:50.904984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:51.053825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091a6-1b13-4885-b2c9-e3c947fe0ea3 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:51.100564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:51.101225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:51.101503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:51.101813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:51.115723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:51.117399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:51.135577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:51.137869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:51.140029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:51.140029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:51.140029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:51.142745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 608/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:51.144778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:51.303173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574713-b7f4-4b8f-b759-f3eb0738a9c3 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:51.307741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:51.362145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:51.363700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:51.364585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:51.365255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:51.366695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:51.367512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:51.370895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:51.396095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 621/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:39:51.402444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:51.415071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:51.583012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a96035-7cfd-46e9-8d35-5c487ef7daf1 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:51.642569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:51.643624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:51.644058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:51.644540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:51.656268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:51.658825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:39:51.683405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:51.685500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:51.687528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 609/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:51.696943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:51.889170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f28a34-04d1-4af8-a391-f66c8bf21eb7 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:51.953851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:51.954650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:51.955009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:51.955440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:51.956451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:51.956850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:51.958664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc271d3a-636e-44da-9dcc-1b79fd7eeff5 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:51.984374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 622/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:39:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:51.985795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:52.101782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:52.173918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd979-b76d-4c15-ab2f-32461a616283 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:52.243459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:52.244385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:52.244754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:52.245207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/8fd64b822c934dada9d93d8179e374ec/roles/538cbcf2da854fe3ba685d968e0dde07` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:52.272598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=8fd64b822c934dada9d93d8179e374ec, role_id=538cbcf2da854fe3ba685d968e0dde07)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:52.274729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912a0d-f67d-4971-ab89-d7f0dbebff7e None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:52.292015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 610/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:39:51 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/8fd64b822c934dada9d93d8179e374ec/roles/538cbcf2da854fe3ba685d968e0dde07 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:39:54.725448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b227-a38a-4957-bea8-14345cef62fe None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:54.726146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b227-a38a-4957-bea8-14345cef62fe None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:54.726146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4b227-a38a-4957-bea8-14345cef62fe None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:54.727754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 623/1233] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:39:54.736919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:54.737715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:54.738053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:54.774596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:39:54.776477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:39:54.879922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 611/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:39:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:39:55.184276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4b227-a38a-4957-bea8-14345cef62fe None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:55.217400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:55.337536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4b227-a38a-4957-bea8-14345cef62fe None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:55.386669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:55.388718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:55.388718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:55.388718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:55.401618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:39:55.406992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:55.409084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:55.410590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 624/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 6 11:39:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:55.416435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:55.522685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:55.616446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aafd8-d1d0-4580-8eb2-f2fe9928b9fa None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:55.670910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:55.671958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:55.672471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:55.672991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:55.674220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] RBAC: Adding query filter params (name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:55.674765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:55.677343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:55.696985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 612/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 6 11:39:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:55.699231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:55.858864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:55.879688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7085-e9d9-4703-b55a-a84e076d0c70 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:55.928183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:55.929116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:55.929426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:55.929898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:55.945452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:55.947035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:55.964556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:55.966329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:55.968108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:55.968108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:55.968108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:55.968108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 625/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:55.976561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:56.152866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a0ed-7f8c-4801-aaba-b26753e7f349 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:56.168315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:56.202921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:56.204495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:56.204825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:56.205106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:56.205911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:56.206312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:56.208294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:56.227591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 613/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:56.235269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:56.380546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42952b-3fe8-4011-be24-0b919ebc7023 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:56.451330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:56.452260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:56.452702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:56.453195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:56.458645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:56.466850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:56.469459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:39:56.483104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:56.486201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:39:56.488207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 626/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:56.491300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:56.636811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990c1d-f834-44f8-8824-9d351f64f665 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:56.684892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:56.685872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:56.686255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:56.686651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:56.687496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:56.688384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:56.690452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:56.713598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 614/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:39:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:56.724061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:56.866252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3ccfb-cd8f-41c9-9b5e-5a0a10bdf522 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:56.910053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311459-6933-44bd-afab-74911681323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:56.911021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311459-6933-44bd-afab-74911681323a None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:56.911296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311459-6933-44bd-afab-74911681323a None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:56.911572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311459-6933-44bd-afab-74911681323a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:57.019679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:57.032983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a311459-6933-44bd-afab-74911681323a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:57.035866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a311459-6933-44bd-afab-74911681323a None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:57.228388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 627/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:39:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:39:57.235170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:57.277745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:57.415556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2803333-fbb5-4321-8112-b104ed4046c0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:57.487305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:57.489757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:57.489757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:57.489757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:57.502023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:57.503825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:39:57.523270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:57.533924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:39:57.536044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:39:57.536044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:39:57.536044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:57.536044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 615/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:39:57.536541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311459-6933-44bd-afab-74911681323a None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:57.698691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:57.708409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311459-6933-44bd-afab-74911681323a None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:57.764378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:57.765274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:57.765739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:57.766924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:57.767827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:57.768249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:57.770037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:57.791200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 628/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:57.801259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:57.914951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:39:57.943242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932723-76a0-4c21-b05e-e82e9a179385 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:57.996603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:57.997499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:57.998116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:57.998647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:58.012984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:58.014706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:39:58.037061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:39:58.039110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:39:58.041299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 616/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:39:58.045484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:58.190051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d6141-05ce-41ef-8dcb-5737f1279bfa None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:58.236512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:58.237593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:58.238149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:58.238815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:58.240209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:58.240833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:58.243972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c5dd-86bd-4095-8a05-c11664007070 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:58.270423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 629/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:39:58.277393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:39:58.447422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd69ef-aa90-4c62-9f57-37aeeb2fce55 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:39:58.503002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:39:58.503650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:39:58.503945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:39:58.504183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:39:58.504969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] RBAC: Adding query filter params (role.id=538cbcf2da854fe3ba685d968e0dde07, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=8fd64b822c934dada9d93d8179e374ec) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:39:58.505306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:39:58.506826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de48af9-7e45-4380-b26d-52b5fa767b26 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:39:58.514068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 617/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:39:58 2022] GET /identity/v3/role_assignments?user.id=8fd64b822c934dada9d93d8179e374ec&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=538cbcf2da854fe3ba685d968e0dde07 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:00.952594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1813a-c1ab-4f19-8a5e-5b4433fffc6e None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:00.954211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1813a-c1ab-4f19-8a5e-5b4433fffc6e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:00.954876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1813a-c1ab-4f19-8a5e-5b4433fffc6e None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:00.956823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 630/1247] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:40:00.963763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:00.964623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:00.964789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:00.975187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:00.993852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:00.996192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:01.103222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 618/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:40:01.392726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1813a-c1ab-4f19-8a5e-5b4433fffc6e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:01.420893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:01.563856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1813a-c1ab-4f19-8a5e-5b4433fffc6e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:01.625979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:01.627004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:01.627306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:01.627564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:01.643332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:40:01.648947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:01.650936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:01.652592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 631/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:40:01.662240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:01.829629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8229f3-5b37-4d73-8fb6-2828657bb195 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:01.876743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:01.883841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:01.884647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:01.885036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:01.885420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:01.886505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:01.886918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:01.888465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:01.903838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 619/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:01.912578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:02.047950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:02.087622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f809a26-5c56-400d-9c1b-369c104d4519 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:02.137833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:02.138886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:02.139355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:02.139849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:02.156094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:02.160117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:02.189679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:02.198966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:02.201069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:02.201069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:02.201069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:02.201069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 632/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:02.201603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:02.365155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f951d4-1a49-4aa0-9dc8-45c93bc68c64 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:02.429832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:02.430874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:02.431355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:02.431789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:02.432539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:02.433063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:02.435210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:02.458943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 620/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:02.463718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:02.639580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeccf9a-6bd4-456d-a19e-2aca86e41992 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:02.695068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:02.705718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:02.706799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:02.707284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:02.707775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:02.720570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:02.722574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:40:02.747419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:02.748473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:02.749493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:02.750858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 633/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:02.761500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:02.841332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:02.916868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511e6b-487d-4fc2-b654-880802c04004 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:02.970470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:02.971451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:02.971910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:02.972331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:02.973224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:02.973598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:02.975511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:03.009309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 621/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:40:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:40:03.018654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:03.160081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58994914-4131-40ee-a633-17e4ee5cbd40 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:03.218556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:03.219741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:03.220231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:03.220694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:03.301707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:03.327821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:03.329795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:03.477854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 634/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:40:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:40:03.486687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:03.691757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305253c-2b79-4171-aecd-d7c4dae86b08 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:03.729771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:03.767087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:03.767795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:03.768135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:03.768492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:03.784227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:03.785787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:03.806394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:03.809176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:03.810937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:03.810937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:03.810937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:03.810937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 622/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:03.815094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:03.949816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:03.966414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad103-52ee-46b2-aad7-905aa477680e None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:04.030130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:04.031002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:04.032118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:04.032563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:04.033675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:04.034068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:04.035627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:04.061068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 635/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:40:04.063739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:04.254835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6aef-35eb-4325-b22d-b9456c0ee345 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:04.317292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:04.318126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:04.318590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:04.319531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:04.332383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:04.334290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:40:04.353833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:04.355505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:04.356470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 623/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:40:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:04.365385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:04.519709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23782c4d-8bc9-4f51-a9a9-3c085a6a3a01 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:04.580416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:04.581432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:04.581880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:04.582380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:04.583619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:04.584329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:04.588928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0d93-82c2-4749-a60f-fbe861b01a7e None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:04.608162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 636/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:40:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:04.619778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:04.664344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:04.806393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7be55-3c95-4f65-bc60-43e02eaa05bf None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:04.854181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:04.854709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:04.854977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:04.855259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:04.856108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=8fd64b822c934dada9d93d8179e374ec) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:04.856646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:04.858742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78339d03-e3d3-4844-8b4e-48fe99da87c1 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:04.866811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 624/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:40:04 2022] GET /identity/v3/role_assignments?user.id=8fd64b822c934dada9d93d8179e374ec&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:07.268858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf29f-9f5b-424a-a896-eff6a66c885a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:07.269435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf29f-9f5b-424a-a896-eff6a66c885a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:07.269911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf29f-9f5b-424a-a896-eff6a66c885a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:07.273678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 637/1261] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:40:07.280332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:07.280876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:07.281857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:07.303687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:07.304999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:07.401831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 625/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:40:07.815798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf29f-9f5b-424a-a896-eff6a66c885a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:07.830256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:07.959811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf29f-9f5b-424a-a896-eff6a66c885a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:08.020820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:08.021583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:08.021994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:08.022443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:08.036332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:40:08.041321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:08.042922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:08.044236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 638/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:40:08.049673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:08.074687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:08.172562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913723-7cc5-4454-87ec-91ad72e23d95 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:08.213706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:08.214359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:08.214641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:08.214916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:08.215615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:08.215921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:08.217350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:08.227276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 626/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:40:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:08.233434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:08.316139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:08.356842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bdb4-1375-4087-b442-f69e3c8f5a84 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:08.394645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:08.395292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:08.395566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:08.395833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:08.409032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:08.410694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:08.426689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:08.428592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:08.430450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:08.430450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:08.430450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:08.430450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 639/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:08.434356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:08.531014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:08.554261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858fa0-adf7-4269-bb43-5050a3a0fec0 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:08.603416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:08.604145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:08.604610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:08.605095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:08.606214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:08.606743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:08.610011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:08.626887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 627/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:40:08.632925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:08.751162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895bd60-5d7a-4d4b-a482-2d6ef147ed3c None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:08.776611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:08.794824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:08.795858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:08.796262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:08.797318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] PATH_INFO: `/v3/users/placement` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:08.811169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:08.813603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:40:08.835482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:08.836736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:08.837961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:08.841835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 640/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:08.847195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:08.963607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7098-a307-49e6-9338-d106dd830e22 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:09.001850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:09.002473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:09.002749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:09.003019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:09.003769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:09.004075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:09.005549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:09.016555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:09.027392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 628/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:40:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:09.036796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:09.190096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f001ca5-bb78-4598-a268-1ec05ad312df None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:09.233346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:09.234014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:09.235031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:09.235419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:09.250475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:09.252399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:09.284377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:09.286505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:09.288539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:09.288539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:09.288539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:09.288539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 641/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:09.294184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:09.417070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7881da9-8ad0-4f97-a8ac-06bf92579555 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:09.457030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:09.457816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:09.457967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:09.458207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:09.458868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:09.459168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:09.460570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:09.473419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 629/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:09.481480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:09.518047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:09.650562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834771b-d91e-440b-a1fa-bbcd73fce710 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:09.712092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:09.713147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:09.713636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:09.714142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:09.727337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:09.729695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:40:09.752612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:09.754434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:09.755714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 642/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:09.760757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:09.827622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:09.877559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e8224-aceb-4c92-a899-18005c4bdb96 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:09.917550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:09.918328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:09.918650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:09.919028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:09.919818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:09.920169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:09.922259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541c0e-abf8-45e8-9475-6e292a6cf344 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:09.936130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 630/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:09.942724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:10.063482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:10.075697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807768-12e4-472c-b4e8-72662d10af33 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:10.139121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:10.140106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:10.141249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:10.141763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] PATH_INFO: `/v3/projects/5695f5deeba84fb2b220872b905c6201/users/8fd64b822c934dada9d93d8179e374ec/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:10.145121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] RBAC: Authorizing `identity:create_grant(project_id=5695f5deeba84fb2b220872b905c6201, user_id=8fd64b822c934dada9d93d8179e374ec, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:10.147760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10696e-e222-42f0-87d2-28ed05590d16 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:10.169886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 643/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 6 11:40:09 2022] PUT /identity/v3/projects/5695f5deeba84fb2b220872b905c6201/users/8fd64b822c934dada9d93d8179e374ec/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:40:12.212903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdc86-3626-4e03-a1a7-35ed52792457 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:12.213476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdc86-3626-4e03-a1a7-35ed52792457 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:12.213476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdc86-3626-4e03-a1a7-35ed52792457 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:12.215055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 631/1274] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:40:12.221594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:12.222170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:12.222261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:12.243817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:12.245254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:12.325682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 644/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:40:12.603063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdc86-3626-4e03-a1a7-35ed52792457 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:12.749438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdc86-3626-4e03-a1a7-35ed52792457 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:12.774087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:12.806307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:12.807173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:12.807628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:12.808080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:12.824030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 11:40:12.829612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:12.831320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:12.832263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 632/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 6 11:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:40:12.840130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:12.944509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:12.967598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22c994-eb1c-418d-a8ac-4d6dfa287019 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.008105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.008740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.009020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.009292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:13.010081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:13.010336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:13.012019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:13.022726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 645/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 6 11:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:13.028740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:13.152583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bfb70-d13f-40e2-95bd-8a0b94604e7c None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.195654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.196213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.196488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.196756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:13.209528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:13.211074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:13.227731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:13.230089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:13.231845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:13.231845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:13.231845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:13.231845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 633/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:13.235766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:13.355818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94adb93-030b-4084-abe7-0087246288ca None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.393250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.393901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.394182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.394455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:13.395116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:13.395422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:13.396817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:13.410504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 646/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:13.416508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:13.453044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:13.533527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc9a3-0af2-495a-8467-a5556480cf8d None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.573841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.574450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.574813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.575084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:13.586658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:13.588245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 11:40:13.604751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:13.605949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:13.607673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 634/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:13.613751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:13.618203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:13.757570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398093c1-d090-4a33-9181-ae0f2cd2365b None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.799579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.800205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.800482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.800750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:13.801503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:13.801949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:13.803518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:13.824416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 647/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 6 11:40:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:13.831336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:13.939126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:13.950965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ab05-be58-4dd7-90af-de00bab0f655 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:13.990516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:13.991235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:13.991512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:13.991782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:14.073562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:14.074995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:14.181337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:14.195141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 635/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 6 11:40:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 11:40:14.201420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:14.285198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:14.360159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ed189-67d2-4781-bf9c-86c1d4e55c74 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:14.422472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:14.423875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:14.424450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:14.424883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:14.445525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:14.448217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 11:40:14.476660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:14.477942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 11:40:14.484408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 11:40:14.484408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 11:40:14.484408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:14.484408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 648/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:40:14.485420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:14.601913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a227-26ef-435e-8c01-823f43a6f3f8 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:14.642610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:14.643141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:14.643418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:14.643690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:14.644350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:14.644654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:14.646137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:14.660411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 636/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 6 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:40:14.666564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:14.791020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca655777-323e-4d0a-a8b6-c8462e6df7ec None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:14.807361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:14.840176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:14.841181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:14.841636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:14.842101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:14.853784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:14.855414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 11:40:14.869040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:14.870182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 11:40:14.872132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 649/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 6 11:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 11:40:14.878226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:14.936741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:14.998356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc705d-a96b-4bd9-a6a1-011fbb9309a7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:15.040699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:15.041333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:15.041613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:15.041932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:15.042661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:15.042975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:15.044480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c65b-435e-4500-bdd7-526e0fc7d699 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:15.059263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 637/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 6 11:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:15.066361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:15.205194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0145ad-f688-4c41-bfdc-8bddbefd28ea None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:15.245993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:15.246622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:15.246902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:15.247170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:15.247996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] RBAC: Adding query filter params (role.id=edae49896b104fddabf7a02644f5d914, scope.project.id=5695f5deeba84fb2b220872b905c6201, user.id=8fd64b822c934dada9d93d8179e374ec) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:15.248300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:15.250709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44023f6-b6ad-44b0-a2e9-10d1f05fc666 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:15.261369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 650/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 6 11:40:15 2022] GET /identity/v3/role_assignments?user.id=8fd64b822c934dada9d93d8179e374ec&scope.project.id=5695f5deeba84fb2b220872b905c6201&role.id=edae49896b104fddabf7a02644f5d914 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:17.371345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75773-fde7-44e6-bb4c-71e078e22c43 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:17.371918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75773-fde7-44e6-bb4c-71e078e22c43 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:17.371918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75773-fde7-44e6-bb4c-71e078e22c43 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:17.373338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 638/1288] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:40:17.380725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:17.381316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:17.381737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:17.406322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:17.407854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:17.445004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:17.516890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 651/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:40:18.015784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75773-fde7-44e6-bb4c-71e078e22c43 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:18.135884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75773-fde7-44e6-bb4c-71e078e22c43 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:18.150383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:18.178782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:18.179455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:18.179734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:18.180075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] PATH_INFO: `/v3/services/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:18.180691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:18.182159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:18.192997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:18.195008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:18.196628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 639/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:40:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:18.200858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:18.329502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfab701-c47a-41dd-be5a-7bc8af0aa224 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:18.374087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:18.375308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:18.375773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:18.376227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:18.376894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] RBAC: Adding query filter params (name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:18.377200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:18.378680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48e5b-6c0c-42b1-ab2b-2df9e3f0fbea None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:18.384581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 652/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 6 11:40:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:18.394587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:18.517604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf0f06-bdce-4d41-80a2-c01c53dec89a None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:18.555236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:18.555933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:18.556327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:18.556649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] PATH_INFO: `/v3/services` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:18.557342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:18.557713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:18.559199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec52b9-a532-4c14-aac9-113d7743abe6 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:18.565475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 640/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 6 11:40:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:20.576825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa6e3-9aa8-48c6-b170-a0d082fb8ea4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:20.577955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa6e3-9aa8-48c6-b170-a0d082fb8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:20.577955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa6e3-9aa8-48c6-b170-a0d082fb8ea4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:20.579392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 653/1293] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:40:20.586174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f69f9-8f18-4ef3-ad93-966622cf3856 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:20.586750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f69f9-8f18-4ef3-ad93-966622cf3856 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:20.586750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f69f9-8f18-4ef3-ad93-966622cf3856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:20.612149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-092f69f9-8f18-4ef3-ad93-966622cf3856 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:20.613781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-092f69f9-8f18-4ef3-ad93-966622cf3856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:20.663243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:20.731949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 641/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:40:21.025816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fa6e3-9aa8-48c6-b170-a0d082fb8ea4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:21.062494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:21.153628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fa6e3-9aa8-48c6-b170-a0d082fb8ea4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:21.193655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:21.194352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:21.194676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:21.194991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:21.195708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:21.197173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743982e-802d-4d07-a70d-2000f1fd6778 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:21.215454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 654/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 6 11:40:21 2022] POST /identity/v3/services => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:40:23.286312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2f55-6ffe-4c54-8cb9-aacf788c0db5 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:23.286972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2f55-6ffe-4c54-8cb9-aacf788c0db5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:23.286972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2f55-6ffe-4c54-8cb9-aacf788c0db5 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:23.288262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 642/1296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:40:23.294476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:23.295361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:23.295361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:23.315376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:23.316643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:23.400952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 655/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 11:40:23.691931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c2f55-6ffe-4c54-8cb9-aacf788c0db5 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:23.875692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c2f55-6ffe-4c54-8cb9-aacf788c0db5 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:23.940358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:23.941434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:23.942023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:23.942439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:23.943484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:23.946095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:23.963832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:23.965314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:23.970398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 643/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:40:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:23.972923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:23.994336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:24.092243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06763df7-a7ab-41b4-98c2-9fd970492d5b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:24.133165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:24.133837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:24.134288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:24.134570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:24.135376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:24.135685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:24.137525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:24.144670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 656/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 6 11:40:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:24.151266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:24.187292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:24.269334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a377296-dac5-489f-825c-5058a8ab14e7 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:24.307257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:24.307979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:24.308131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:24.308380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:24.309096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] RBAC: Adding query filter params (interface=public, service_id=3626659068dc4427b1000f6585b9ea41, region_id=RegionOne) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:24.309400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:24.310846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686760-dd83-48e6-8601-3c2bc80f5d9c None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:24.317380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 644/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 6 11:40:24 2022] GET /identity/v3/endpoints?service_id=3626659068dc4427b1000f6585b9ea41&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:24.323568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:24.420636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:24.442749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d281c4d-6e11-4f7a-be82-91caa593a9d8 None admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:24.481185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:24.481843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:24.482139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:24.482399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:24.483024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:24.485442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a931-fd0b-45ad-899d-9d0b28541179 None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:24.493244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 657/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 6 11:40:24 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:40:26.545878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368523bd-1373-444b-a6ba-70c288a5d54c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:26.546649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368523bd-1373-444b-a6ba-70c288a5d54c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:26.546649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368523bd-1373-444b-a6ba-70c288a5d54c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:26.547776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 645/1302] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:40:26.554976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:26.555584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:26.555584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:26.580999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:40:26.582543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:40:26.685859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 658/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:40:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 11:40:26.962152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368523bd-1373-444b-a6ba-70c288a5d54c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:27.073724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:27.081999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368523bd-1373-444b-a6ba-70c288a5d54c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:27.122827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:27.123646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:27.123967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:27.124737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:27.125447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:27.126957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 11:40:27.137345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 11:40:27.139252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application with Lock( Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self._enter() Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application return self.creator() Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application created_value = creator( Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: ERROR keystone.server.flask.application Sep 06 11:40:27.142127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 646/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 6 11:40:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 11:40:27.147773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:27.233938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:27.261807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ccc17-1aba-411f-aac3-5407b35a251e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:27.299115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:27.299733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:27.300005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:27.300271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] PATH_INFO: `/v3/services` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:27.300910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] RBAC: Adding query filter params (name=placement) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:40:27.301212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:27.302651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0424a1b-0a13-4ea2-be21-cd49bf69f3af None admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:27.308661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 659/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 6 11:40:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:40:27.315702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:40:27.433245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd479c-4d3f-4fab-a21f-db20927b6456 None admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:40:27.473236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:40:27.473953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:40:27.474297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:40:27.474638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:40:27.475427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:40:27.476900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109c20f-9f15-4fc3-a6ea-c0608fe1cc2d None admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:40:27.501250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 647/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:40:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:40:37.305176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=30 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:40:37.495070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:43.526319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e5ab-e985-435c-91cf-5e5ec35220ad None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:43.527178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e5ab-e985-435c-91cf-5e5ec35220ad None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:43.527178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e5ab-e985-435c-91cf-5e5ec35220ad None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:43.528765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 660/1307] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:41:43.535155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a7b2-5eab-449f-b1c8-bad8032242a4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:43.536006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a7b2-5eab-449f-b1c8-bad8032242a4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:43.536006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240a7b2-5eab-449f-b1c8-bad8032242a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:43.634087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5240a7b2-5eab-449f-b1c8-bad8032242a4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:43.635359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5240a7b2-5eab-449f-b1c8-bad8032242a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:43.945747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 648/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:41:44.285621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:44.285937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:44.286207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:44.288260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 661/1309] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 6 11:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:41:44.302578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852294c-de71-4510-a69d-21c6b7f4acc4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:44.304186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852294c-de71-4510-a69d-21c6b7f4acc4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:44.304186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852294c-de71-4510-a69d-21c6b7f4acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:44.423676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d852294c-de71-4510-a69d-21c6b7f4acc4 None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:44.425913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d852294c-de71-4510-a69d-21c6b7f4acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:44.466737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:44.552684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 649/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 6 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:41:44.563221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:41:44.566643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:44.787604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f433c5-f3c1-4a3e-ac74-e5f8fcf23d57 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:41:44.978723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74809e26-7812-4378-a090-e34326131edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:41:44.981844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809e26-7812-4378-a090-e34326131edb service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:44.982859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809e26-7812-4378-a090-e34326131edb service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:44.983346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809e26-7812-4378-a090-e34326131edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:45.001866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-74809e26-7812-4378-a090-e34326131edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:45.167830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:45.293870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74809e26-7812-4378-a090-e34326131edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:41:45.296211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74809e26-7812-4378-a090-e34326131edb service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:41:45.321099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 662/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:41:48.328642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:48.329208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:48.329208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:48.330577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 650/1312] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:41:48.336476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1458-03ae-4f51-ac7a-389c09eb994e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:48.337016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1458-03ae-4f51-ac7a-389c09eb994e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:48.337016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac1458-03ae-4f51-ac7a-389c09eb994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:48.432697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f6ac1458-03ae-4f51-ac7a-389c09eb994e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:48.434017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f6ac1458-03ae-4f51-ac7a-389c09eb994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:48.568801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 663/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:41:48.870241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:41:48.871964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:48.991958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddb896-d3be-4ea9-8af2-235326f928b4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:41:49.076834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:49.079923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:41:49.080965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:49.081877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:49.082089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:49.084115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:49.209265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:41:49.210752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401529f9-ad0b-4c45-95f4-4c38ef4dac88 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:41:49.232097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 651/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:41:51.651989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:51.652594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:51.652594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:51.654107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 664/1315] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:41:51.660987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb4e9-8b6e-4a1f-a0d6-bf67aa037252 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:51.661778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb4e9-8b6e-4a1f-a0d6-bf67aa037252 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:51.661778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb4e9-8b6e-4a1f-a0d6-bf67aa037252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:51.741354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-881eb4e9-8b6e-4a1f-a0d6-bf67aa037252 None None] MFA Rules not processed for user `2c22fa2cde544208ab6c0e4a6f861028`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:51.742663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-881eb4e9-8b6e-4a1f-a0d6-bf67aa037252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:51.804569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:51.887703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 652/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:41:52.182065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:41:52.184187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:52.195230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:52.307011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b7ae-2240-4e37-a1de-e8f88dd14957 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:41:52.394448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:41:52.395065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:52.395341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:52.395609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:52.397271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:52.465405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:52.517998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:41:52.519448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503044a-c2fe-485c-929c-43369a291fac service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:41:52.539635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 665/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:41:55.291114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04dc49-dd6b-481a-8188-14058fc2bd87 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:55.291695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04dc49-dd6b-481a-8188-14058fc2bd87 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:55.291695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04dc49-dd6b-481a-8188-14058fc2bd87 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:55.293095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 653/1318] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:41:55.299311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f517dd-2f05-4c26-b658-4ac3d0e62e1b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:55.299962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f517dd-2f05-4c26-b658-4ac3d0e62e1b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:55.299962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f517dd-2f05-4c26-b658-4ac3d0e62e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:55.386218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d3f517dd-2f05-4c26-b658-4ac3d0e62e1b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:55.388080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d3f517dd-2f05-4c26-b658-4ac3d0e62e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:55.518414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 666/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:41:55.834338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:55.834631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:55.834866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:55.836284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 654/1320] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 6 11:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:41:55.844219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cd0d1-ac28-4672-bc34-aeac3253406e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:55.844502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cd0d1-ac28-4672-bc34-aeac3253406e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:55.844780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cd0d1-ac28-4672-bc34-aeac3253406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:55.870127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:55.943018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-972cd0d1-ac28-4672-bc34-aeac3253406e None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:41:55.944282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-972cd0d1-ac28-4672-bc34-aeac3253406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:56.037688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 667/1321] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 6 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:41:56.048556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:41:56.050192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:56.198537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77483ada-7310-46c0-8d69-c9545c7381e3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:41:56.290174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:41:56.291018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:56.291427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:56.292013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:56.294391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:41:56.303331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:41:56.429488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:41:56.431059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aec82-91f1-405d-bd1e-22e9fce625a7 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:41:56.459960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 655/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:41:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:41:59.937129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:59.937801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:59.938150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:41:59.940600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 668/1323] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:41:59.948650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bece-c847-430a-b662-9c2d61d6dfd3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:41:59.949895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bece-c847-430a-b662-9c2d61d6dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:41:59.950035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bece-c847-430a-b662-9c2d61d6dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:00.032093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3fa1bece-c847-430a-b662-9c2d61d6dfd3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:00.033333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3fa1bece-c847-430a-b662-9c2d61d6dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:00.107652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:00.159838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 656/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:00.465052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:00.467637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:00.597778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b298f-9ca9-41d6-84d1-7c0f15cb112a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:00.612006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:00.688758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:00.689378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:00.689672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:00.689955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:00.691632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:00.836059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:00.837538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63517776-ac59-43fb-a9f5-309511b85984 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:00.861077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 669/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:03.053732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:03.054530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:03.054530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:03.056080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 657/1326] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:03.063913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65981416-724b-40b6-b38e-c4546f5ed3ab None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:03.064504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65981416-724b-40b6-b38e-c4546f5ed3ab None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:03.065029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65981416-724b-40b6-b38e-c4546f5ed3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:03.148529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:03.174968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-65981416-724b-40b6-b38e-c4546f5ed3ab None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:03.176226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-65981416-724b-40b6-b38e-c4546f5ed3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:03.304782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 670/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:03.605745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:03.607391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:03.736865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5d81-989a-4d37-a660-3dda3f1f80c2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:03.831897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:03.832516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:03.832801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:03.833076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:03.834869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:03.856516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:03.977787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:03.979311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2cae6-1750-47e3-a85c-5283fc4209d7 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:04.003224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 658/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:06.207594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:06.208495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:06.208495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:06.209515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 671/1329] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:06.218731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe29b-12e5-4282-8b39-5ebbeaa7801e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:06.219472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe29b-12e5-4282-8b39-5ebbeaa7801e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:06.219677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe29b-12e5-4282-8b39-5ebbeaa7801e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:06.331701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ff4fe29b-12e5-4282-8b39-5ebbeaa7801e None None] MFA Rules not processed for user `2c22fa2cde544208ab6c0e4a6f861028`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:06.333165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ff4fe29b-12e5-4282-8b39-5ebbeaa7801e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:06.429596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:06.473097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 659/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:06.823077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:06.826778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:07.022688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1399b-10ff-4d53-b277-d2c9c85c98a3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:07.051350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:07.145148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:07.146195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:07.146707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:07.147190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:07.150476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:07.323718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:07.326313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e7a56-1159-4a88-9b2b-fabd4d3cec51 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:07.361358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 672/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:10.245776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:10.246611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:10.247059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:10.248572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 660/1332] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:10.254910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f091-2fab-4afc-be56-64c6e4ba0b4a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:10.255487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f091-2fab-4afc-be56-64c6e4ba0b4a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:10.255487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f091-2fab-4afc-be56-64c6e4ba0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:10.341236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:10.345422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ee90f091-2fab-4afc-be56-64c6e4ba0b4a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:10.347527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ee90f091-2fab-4afc-be56-64c6e4ba0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:10.487141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 673/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:42:10.789721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:10.791198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:10.917539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c647e67-02ae-4a01-849b-2b8d14a7d76a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:11.004297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:11.005048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:11.005410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:11.005893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:11.007815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:11.040714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:11.163546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:11.165199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b500aba-2acb-4a57-af7c-837f9203e3f0 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:11.191164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 661/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:13.342472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:13.343033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:13.343167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:13.344826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 674/1335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:13.351136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2ae8-b345-44fc-8842-840f33ed0bb9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:13.351742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2ae8-b345-44fc-8842-840f33ed0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:13.351742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2ae8-b345-44fc-8842-840f33ed0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:13.430820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-15eb2ae8-b345-44fc-8842-840f33ed0bb9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:13.432126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-15eb2ae8-b345-44fc-8842-840f33ed0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:13.524589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:13.551600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 662/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:13.849628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:13.851231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:13.979753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1579fc-ff7a-44f8-b86e-d0006e272120 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:14.015704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:14.072435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:14.073118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:14.073431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:14.073799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:14.075716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:14.211667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:14.213227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ec2cc-d402-4c15-b800-d0caa938f688 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:14.235804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 675/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:42:16.687939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:16.688571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:16.688571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:16.689892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 663/1338] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:16.696246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236697c-55c7-43da-80e9-8c06eeaa17fd None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:16.696832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236697c-55c7-43da-80e9-8c06eeaa17fd None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:16.696832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236697c-55c7-43da-80e9-8c06eeaa17fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:16.795432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e236697c-55c7-43da-80e9-8c06eeaa17fd None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:16.797593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e236697c-55c7-43da-80e9-8c06eeaa17fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:16.807956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:16.910649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 676/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:17.205510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:17.207151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:17.350470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e20857-5d85-47d4-b564-2f8c07bccfdd None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:17.443832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:17.444453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:17.444731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:17.445006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:17.446768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:17.495414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:17.593443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:17.595143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad53f8-85f6-4dbb-a5f6-c8c7abbccd38 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:17.619335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 664/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:42:22.184594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:22.185259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:22.185541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:22.188155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 677/1341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:22.196601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afde35-fc5f-4ddf-8167-1c111c086986 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:22.197435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afde35-fc5f-4ddf-8167-1c111c086986 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:22.197435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afde35-fc5f-4ddf-8167-1c111c086986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:22.301004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-90afde35-fc5f-4ddf-8167-1c111c086986 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:22.302583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-90afde35-fc5f-4ddf-8167-1c111c086986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:22.420177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:22.436080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 665/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:22.734643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:22.736206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:22.881430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509d35c-4797-4734-8d93-50fbb1b93d65 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:22.930305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:22.984648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:22.985499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:22.986016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:22.986414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:22.988292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:23.174571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:23.176055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c03185-b97b-4df1-a7a8-9fee7b408218 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:23.197713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 678/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:26.056308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:26.056984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:26.056984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:26.059264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 666/1344] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:26.065538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55981b-0675-4977-966a-b5b8a3d6a671 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:26.066201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55981b-0675-4977-966a-b5b8a3d6a671 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:26.066201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55981b-0675-4977-966a-b5b8a3d6a671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:26.149374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-bc55981b-0675-4977-966a-b5b8a3d6a671 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:26.150686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-bc55981b-0675-4977-966a-b5b8a3d6a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:26.172983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:26.287951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 679/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:26.590082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:26.591790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:26.756253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062eb6d-c5b5-418d-a502-9f373149a37a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:26.862455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:26.862961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:26.863242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:26.863507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:26.865200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:26.950718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:27.083166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:27.085520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc5cc7-817e-4273-bc3a-03085dc621e2 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:27.107661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 667/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:42:32.888662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:32.889297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:32.889601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:32.891644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 680/1347] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:32.899588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf59-048b-449a-a986-f6de671047b7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:32.900265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf59-048b-449a-a986-f6de671047b7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:32.900543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf59-048b-449a-a986-f6de671047b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:33.014704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b44edf59-048b-449a-a986-f6de671047b7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:33.015860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b44edf59-048b-449a-a986-f6de671047b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:33.174289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:33.181786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 668/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:33.477301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:33.479459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:33.643768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4766917-1bc6-4907-9748-185f4ceeca3e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:33.698667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:33.746964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:33.747885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:33.748175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:33.748444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:33.750218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:33.888367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:33.889911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727658-6311-4c0d-9a3e-b134ea3a7b00 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:33.911404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 681/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:42:40.043558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e78d1-2c01-471d-b9ac-b8d1180e6f97 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:40.044311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e78d1-2c01-471d-b9ac-b8d1180e6f97 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:40.047757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e78d1-2c01-471d-b9ac-b8d1180e6f97 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:40.049336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 669/1350] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:42:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:42:40.059580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be5b7-925d-43a9-aba2-3578e5f3dcb0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:40.061113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be5b7-925d-43a9-aba2-3578e5f3dcb0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:40.061113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be5b7-925d-43a9-aba2-3578e5f3dcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:40.281228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-416be5b7-925d-43a9-aba2-3578e5f3dcb0 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:40.284334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-416be5b7-925d-43a9-aba2-3578e5f3dcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:40.330768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:40.549236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 682/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:46.078076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe0f3-cb6b-4475-8294-3aa25fcad5bd None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:46.078983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe0f3-cb6b-4475-8294-3aa25fcad5bd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:46.079442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbe0f3-cb6b-4475-8294-3aa25fcad5bd None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:46.081361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 670/1352] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:42:46.088780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e8ba-6bc5-425a-ab85-16f9458212d9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:46.089396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e8ba-6bc5-425a-ab85-16f9458212d9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:46.092234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e8ba-6bc5-425a-ab85-16f9458212d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:46.222660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4f73e8ba-6bc5-425a-ab85-16f9458212d9 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:46.224355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4f73e8ba-6bc5-425a-ab85-16f9458212d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:46.341759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:46.397934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 683/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:42:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:48.176618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b7a2-ef64-41a9-8e1d-19f551eed4d8 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:48.177411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b7a2-ef64-41a9-8e1d-19f551eed4d8 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:48.177411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129b7a2-ef64-41a9-8e1d-19f551eed4d8 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:48.179530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 671/1354] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:48.185897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1a6f-c4ff-4e03-90a5-44bebfd242e2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:48.186713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1a6f-c4ff-4e03-90a5-44bebfd242e2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:48.187007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b1a6f-c4ff-4e03-90a5-44bebfd242e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:48.299834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a11b1a6f-c4ff-4e03-90a5-44bebfd242e2 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:48.301188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a11b1a6f-c4ff-4e03-90a5-44bebfd242e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:48.443897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 684/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:42:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:52.358394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ccc9-4387-43be-bafe-24c6ea639682 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:52.359145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ccc9-4387-43be-bafe-24c6ea639682 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:52.359145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ccc9-4387-43be-bafe-24c6ea639682 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:52.360338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 672/1356] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:52.367167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44c07-257a-41ac-a864-6b6e632699a3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:52.367965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44c07-257a-41ac-a864-6b6e632699a3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:52.368129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44c07-257a-41ac-a864-6b6e632699a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:52.418089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:52.460775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e2d44c07-257a-41ac-a864-6b6e632699a3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:52.462068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e2d44c07-257a-41ac-a864-6b6e632699a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:52.637566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 685/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:53.218105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:53.219192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:53.220135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:53.224976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 673/1358] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 6 11:42:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:53.240457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887abde-af30-4931-9c45-46ef206f3244 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:53.241629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887abde-af30-4931-9c45-46ef206f3244 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:53.242875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887abde-af30-4931-9c45-46ef206f3244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:53.393974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b887abde-af30-4931-9c45-46ef206f3244 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:53.395072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b887abde-af30-4931-9c45-46ef206f3244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:53.485223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:53.631397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 686/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 6 11:42:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:53.646629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:53.649105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:53.914014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4ddee-b35f-4265-a0cf-22db192a1876 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:54.005770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:54.058849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:54.059606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:54.060906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:54.061198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:54.063131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:54.177263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:54.179030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409eb97-6904-40f4-9684-9ce5f25f5a18 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:54.204413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 674/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:42:56.889444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:56.890362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:56.890362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:56.892796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 687/1361] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:42:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:42:56.902903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0386b-9592-4b0a-867f-b364a96e590f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:56.904035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0386b-9592-4b0a-867f-b364a96e590f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:56.904412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0386b-9592-4b0a-867f-b364a96e590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:57.063287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2bb0386b-9592-4b0a-867f-b364a96e590f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:42:57.065592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2bb0386b-9592-4b0a-867f-b364a96e590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:57.190527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:57.301170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 675/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:42:57.788150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:42:57.789976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:58.019847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e537ac-909e-4467-8bd7-d75145a015ef None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:42:58.040121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:42:58.279701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:42:58.281360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:42:58.282083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:42:58.282779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:42:58.286856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:42:58.461334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:42:58.463294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87976e1-b8c3-40b1-a4ab-d508f639d791 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:42:58.490038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 688/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:01.876135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b2b5-23da-48ea-b9c6-216059679e45 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:01.876741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b2b5-23da-48ea-b9c6-216059679e45 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:01.877081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b2b5-23da-48ea-b9c6-216059679e45 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:01.879869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 676/1364] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:01.888946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e258-e13e-43c8-b48b-e75c76917508 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:01.889821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e258-e13e-43c8-b48b-e75c76917508 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:01.889821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e258-e13e-43c8-b48b-e75c76917508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:01.909571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:01.999149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ea34e258-e13e-43c8-b48b-e75c76917508 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:02.001229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ea34e258-e13e-43c8-b48b-e75c76917508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:02.153796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 689/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:02.604322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47753b6-232d-4d12-9677-17e87df56c7a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:02.604799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47753b6-232d-4d12-9677-17e87df56c7a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:02.606382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47753b6-232d-4d12-9677-17e87df56c7a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:02.609437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 677/1366] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 6 11:43:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:02.619380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:02.619784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:02.620166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:02.748843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:02.751262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:02.835247 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:02.873628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded4a8-8cb8-4e9e-bafe-84aea646431c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:02.873992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded4a8-8cb8-4e9e-bafe-84aea646431c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:02.874351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded4a8-8cb8-4e9e-bafe-84aea646431c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:02.878153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 678/1367] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:43:02.883802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0fbe-368b-4a75-861e-5f7f50a9cc72 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:02.884236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0fbe-368b-4a75-861e-5f7f50a9cc72 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:02.884599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0fbe-368b-4a75-861e-5f7f50a9cc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:02.980322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 690/1368] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 6 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:02.992155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:02.999245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c4ab0fbe-368b-4a75-861e-5f7f50a9cc72 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:03.000833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c4ab0fbe-368b-4a75-861e-5f7f50a9cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:03.006087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152f2f3-3b08-4bec-95b4-15c49f299cd2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:03.156826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 679/1369] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:03.171058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:03.171878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:03.172403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:03.172783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:03.175159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:03.205075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:03.351681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:03.355776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c649-1bea-43e9-80e4-158410e603e4 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:03.392534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 691/1370] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:04.033396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:04.034416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:04.034633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:04.037383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 680/1371] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 6 11:43:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:43:04.046089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ffe6a-ca45-47cf-933f-ad9e6c67603c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:04.046965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ffe6a-ca45-47cf-933f-ad9e6c67603c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:04.047128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ffe6a-ca45-47cf-933f-ad9e6c67603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:04.219466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b99ffe6a-ca45-47cf-933f-ad9e6c67603c None None] MFA Rules not processed for user `ccafefb49d0c4727bad188927ae5bf7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:04.221403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b99ffe6a-ca45-47cf-933f-ad9e6c67603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:04.403171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 692/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 6 11:43:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:04.412660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:04.415669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:04.456777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:04.644546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f72ab-ab0e-4d08-90bf-9f731d70cfa1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:04.822443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:04.824103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:04.825093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:04.825335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:04.827752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:04.964766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:05.053187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:05.056236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be8b8-1271-4c27-b0a4-d4f85e3cab5e service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:05.111025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 681/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:43:06.043815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:06.044655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:06.044810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:06.046533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 693/1374] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:06.052743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc93ca-6bb6-4f54-bee7-2cfcecf6764b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:06.053293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc93ca-6bb6-4f54-bee7-2cfcecf6764b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:06.054038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc93ca-6bb6-4f54-bee7-2cfcecf6764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:06.162071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bbfc93ca-6bb6-4f54-bee7-2cfcecf6764b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:06.164289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bbfc93ca-6bb6-4f54-bee7-2cfcecf6764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:06.363904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 682/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:06.680164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:06.681849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:06.691570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:06.911957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cc188-9803-4d79-bc7e-2df71760f2e1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:07.071162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:07.143298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:07.144374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:07.144861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:07.145332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:07.148440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:07.317687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:07.320354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571799b7-848f-4556-bc7d-f9e82979bcc9 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:07.351676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 694/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:09.899504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:09.901165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:09.902778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:09.905981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 683/1377] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:09.915767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef125a3-752d-46c6-9820-dc138e52ca52 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:09.916601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef125a3-752d-46c6-9820-dc138e52ca52 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:09.917525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef125a3-752d-46c6-9820-dc138e52ca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:10.063146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-cef125a3-752d-46c6-9820-dc138e52ca52 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:10.065494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cef125a3-752d-46c6-9820-dc138e52ca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:10.161930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:10.285776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 695/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:10.801963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:10.805091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:10.871510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:11.068742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c0fe2-99fe-4f85-859f-5a7c957a8f3c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:11.302899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:11.332784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:11.333777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:11.334282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:11.334760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:11.338084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:11.555497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:11.559344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bf3fe-9e2b-4c1b-afb5-8474bb11508d service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:11.602838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 684/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:13.981979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:13.982544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:13.982948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:13.985694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 696/1380] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:13.995021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479e98-716a-4159-805c-1fb34ea60b51 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:13.996098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479e98-716a-4159-805c-1fb34ea60b51 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:13.996868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479e98-716a-4159-805c-1fb34ea60b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:14.121222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d5479e98-716a-4159-805c-1fb34ea60b51 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:14.123138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d5479e98-716a-4159-805c-1fb34ea60b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:14.229425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:14.288721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 685/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:14.606563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:14.608652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:14.820351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf74d-60b8-4343-9328-5485494aacd1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:14.862446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:15.006208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:15.007314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:15.007794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:15.008270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:15.010822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:15.154725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:15.156550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0c845-85d1-452a-adf3-cfe880011882 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:15.192774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 697/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:43:17.930258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:17.931201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:17.931383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:17.935224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 686/1383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:17.943294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda8a2-1e70-4645-b32b-00968771eaee None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:17.944139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda8a2-1e70-4645-b32b-00968771eaee None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:17.944139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda8a2-1e70-4645-b32b-00968771eaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:18.000697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:18.043648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-53fda8a2-1e70-4645-b32b-00968771eaee None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:18.045272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-53fda8a2-1e70-4645-b32b-00968771eaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:18.189205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 698/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:18.507075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:18.509188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:18.736340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d474265-0584-46a5-97c2-a3686e3000a1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:18.826800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:18.930407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:18.931140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:18.931300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:18.931573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:18.933338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:19.052945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:19.054826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64f28e-e7d9-4a7e-b8dd-d30e40147697 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:19.084771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 687/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:21.307563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:21.308423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:21.308645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:21.310230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 699/1386] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:21.316210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f195d-ed1a-4e33-bbb9-4dbabef91cf0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:21.316818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f195d-ed1a-4e33-bbb9-4dbabef91cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:21.316818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f195d-ed1a-4e33-bbb9-4dbabef91cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:21.435972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-599f195d-ed1a-4e33-bbb9-4dbabef91cf0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:21.438164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-599f195d-ed1a-4e33-bbb9-4dbabef91cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:21.442087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:21.558672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 688/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:21.866213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:21.869124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:21.980200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fc770-49a9-4ec5-bbfc-5527c8a19402 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:21.980951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fc770-49a9-4ec5-bbfc-5527c8a19402 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:21.981133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fc770-49a9-4ec5-bbfc-5527c8a19402 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:21.983216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 689/1388] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:43:21.989018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111b0-7632-4103-8b3e-7d47bea4f349 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:21.989299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111b0-7632-4103-8b3e-7d47bea4f349 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:21.989609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111b0-7632-4103-8b3e-7d47bea4f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:22.006153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:22.082265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-7fa111b0-7632-4103-8b3e-7d47bea4f349 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:22.083544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7fa111b0-7632-4103-8b3e-7d47bea4f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:22.092081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dcdf6-3870-4f25-9c81-9d2998b931f5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:22.176296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:22.201810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 690/1389] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:22.278423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:22.279040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:22.279312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:22.279577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:22.281228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:22.372059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:22.399715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:22.405436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368762e-3bdc-45a8-b1d1-776dcbd062b0 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:22.436632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 700/1390] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:22.550982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:22.551610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:22.551936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:22.554780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 691/1391] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 6 11:43:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:22.563960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84f44-f981-4ed4-8934-2c9027c972d4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:22.564354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84f44-f981-4ed4-8934-2c9027c972d4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:22.564737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84f44-f981-4ed4-8934-2c9027c972d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:22.676098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5cc84f44-f981-4ed4-8934-2c9027c972d4 None None] MFA Rules not processed for user `7739336ec72248feb11d1d6a7a8de166`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:22.677472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5cc84f44-f981-4ed4-8934-2c9027c972d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:22.777501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 701/1392] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 6 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:22.785864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:22.789025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:22.972843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee474a-2009-4808-838c-8c2a40e05656 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:23.071236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:23.071862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:23.072139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:23.072411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:23.074302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:23.123176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:23.212293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:23.213811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605869e-83c8-4e5a-9207-5b87621c3158 service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:23.235563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 692/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:24.204533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2874f9-a692-457c-b91f-2dabc33ac186 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.205454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2874f9-a692-457c-b91f-2dabc33ac186 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.205454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2874f9-a692-457c-b91f-2dabc33ac186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.294715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-eb2874f9-a692-457c-b91f-2dabc33ac186 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:24.296009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-eb2874f9-a692-457c-b91f-2dabc33ac186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:24.319659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:24.433861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 702/1394] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:43:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:24.492950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442bae-7458-4bec-bcd3-4600e0b5e7da None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.493699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442bae-7458-4bec-bcd3-4600e0b5e7da None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.493699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442bae-7458-4bec-bcd3-4600e0b5e7da None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.494902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 693/1395] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:24.496296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5500c7-a47b-4620-b7f3-5ac5a53121d5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.496646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5500c7-a47b-4620-b7f3-5ac5a53121d5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.497024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5500c7-a47b-4620-b7f3-5ac5a53121d5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.498724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 703/1396] 10.222.0.18 () {54 vars in 943 bytes} [Tue Sep 6 11:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:24.501213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.501760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.501760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.507735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.508175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.508622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.577220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:24.578496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:24.603490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] MFA Rules not processed for user `f7848ff43a794bd4aa35bdff59a53c73`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:24.606038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:24.704603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:24.709357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:24.714462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 694/1397] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:24.740638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 704/1398] 10.222.0.18 () {58 vars in 1040 bytes} [Tue Sep 6 11:43:24 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:24.751392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:24.752640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:24.865989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f788df1-e34c-42bf-9ae5-f7dafc36adcf None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:24.982437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:24.982950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:24.983228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:24.983500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:24.985188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:25.001755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:25.015841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:25.017154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:25.096585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:25.097952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0566cc-fb27-4240-9379-06cb62a22a1f service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:25.107270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 695/1399] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:43:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:43:25.146161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8b23a-0c54-41dd-9b6c-c6325359e101 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:25.367990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:25.373905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:25.375001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:25.375524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:25.376294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:25.380336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:25.581902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:25.583465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971c9e5-a5a5-4ba7-935b-ab0e6fa756e5 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:25.608455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 705/1400] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:43:28.058778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:28.059586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:28.059586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:28.061018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 696/1401] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:28.066857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eaf1-8654-4d42-8b20-a3676748e982 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:28.067415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eaf1-8654-4d42-8b20-a3676748e982 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:28.067415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eaf1-8654-4d42-8b20-a3676748e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:28.170439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a006eaf1-8654-4d42-8b20-a3676748e982 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:28.171807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a006eaf1-8654-4d42-8b20-a3676748e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:28.328692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:28.395256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 706/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:28.749875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:28.751494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:28.985065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9431-94c0-43bb-8c6e-bf9ecdc167ca None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:29.000111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:29.180195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:29.181332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:29.182069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:29.183349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:29.187422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:29.367700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:29.369545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93871-60a2-4f7c-b935-b57da03d1614 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:29.398729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 697/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:31.711951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:31.712637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:31.712637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:31.714714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 707/1404] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:31.720549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b95fa-ed27-4c99-af1d-f91f93eb86b7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:31.720549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b95fa-ed27-4c99-af1d-f91f93eb86b7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:31.720549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b95fa-ed27-4c99-af1d-f91f93eb86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:31.739510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:31.816507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-593b95fa-ed27-4c99-af1d-f91f93eb86b7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:31.817784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-593b95fa-ed27-4c99-af1d-f91f93eb86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:31.979411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 698/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:32.343518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:32.345437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:32.583512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e7a3d-7e39-4508-9b1f-d733ec0e0281 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:32.721702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:32.811221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:32.812160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:32.812437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:32.812708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:32.814455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:32.939407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:32.941140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22486c3-403c-4915-9e9d-4231f1066b10 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:32.963187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 708/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:35.033804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:35.034708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:35.034708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:35.038046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 699/1407] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:43:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:35.041318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1e54-ec78-40bb-b743-e231b5e717a6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:35.041871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1e54-ec78-40bb-b743-e231b5e717a6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:35.041871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1e54-ec78-40bb-b743-e231b5e717a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:35.119305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-efdf1e54-ec78-40bb-b743-e231b5e717a6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:35.121796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-efdf1e54-ec78-40bb-b743-e231b5e717a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:35.142980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:35.291544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 709/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:35.596455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:35.598028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:35.706226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:35.802125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248539d-60aa-4851-acfa-adfc41587a3c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:35.970528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:35.971279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:35.971627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:35.971973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:35.974427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:36.067377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:36.112787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:36.114491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e94dfc-afac-49ca-8a69-1acdb3dbc3bd service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:36.140237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 700/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:43:45.289257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=38 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:46.132846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=22 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:48.186924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d29-b02e-4b88-988a-cc4ba80c25a7 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:48.187509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d29-b02e-4b88-988a-cc4ba80c25a7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:48.187509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23704d29-b02e-4b88-988a-cc4ba80c25a7 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:48.189156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 710/1410] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 6 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:48.194972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927ba4-ebf6-4048-b082-785be6d6629d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:48.195503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927ba4-ebf6-4048-b082-785be6d6629d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:48.195885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927ba4-ebf6-4048-b082-785be6d6629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:48.281233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4a927ba4-ebf6-4048-b082-785be6d6629d None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:48.284343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4a927ba4-ebf6-4048-b082-785be6d6629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:48.393788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 701/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 6 11:43:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:48.408447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3054-f318-4860-8cc1-dc0488f496e2 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:48.408726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3054-f318-4860-8cc1-dc0488f496e2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:48.408999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3054-f318-4860-8cc1-dc0488f496e2 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:48.410382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 711/1412] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 6 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:48.415520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674923c7-5cbf-4be9-af9c-6e91149c8137 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:48.415922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674923c7-5cbf-4be9-af9c-6e91149c8137 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:48.416315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674923c7-5cbf-4be9-af9c-6e91149c8137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:48.495896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:48.500157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-674923c7-5cbf-4be9-af9c-6e91149c8137 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:48.502110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-674923c7-5cbf-4be9-af9c-6e91149c8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:48.622424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 702/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 6 11:43:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:51.056898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:51.058015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:51.058015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:51.058835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:51.064075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f90f7-ea51-4102-bf5c-056fd78b10b2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:51.064483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f90f7-ea51-4102-bf5c-056fd78b10b2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:51.064629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f90f7-ea51-4102-bf5c-056fd78b10b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:51.132898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a72f90f7-ea51-4102-bf5c-056fd78b10b2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:51.134167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a72f90f7-ea51-4102-bf5c-056fd78b10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:51.190369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:51.232136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:51.527280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:51.528986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:51.675427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c5ba-0471-4e81-ae43-0ac7009da013 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:51.748866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:51.793635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:51.794198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:51.794473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:51.794764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:51.796379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:51.889291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:51.890935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6994660-c959-4bee-a19e-2affb152f258 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:51.910685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 713/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:52.832139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144870aa-3901-4185-96c6-17dee518e159 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:52.832752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144870aa-3901-4185-96c6-17dee518e159 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:52.832752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144870aa-3901-4185-96c6-17dee518e159 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:52.834134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:52.840358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9985d-fae4-47a4-bf96-0746f1d56281 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:52.840777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9985d-fae4-47a4-bf96-0746f1d56281 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:52.840906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9985d-fae4-47a4-bf96-0746f1d56281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:52.918394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2ce9985d-fae4-47a4-bf96-0746f1d56281 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:52.919627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2ce9985d-fae4-47a4-bf96-0746f1d56281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:52.946045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:53.019879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:53.178533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144870aa-3901-4185-96c6-17dee518e159 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:53.180350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-144870aa-3901-4185-96c6-17dee518e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:53.329551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144870aa-3901-4185-96c6-17dee518e159 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:53.373231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:53.447833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:53.449037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:53.449315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:53.449585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:53.451213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:53.541131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:53.543442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4874b6d-c630-4fe9-b586-e61241ba40b6 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:53.562170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 705/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:54.453494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:54.454081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:54.454081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:54.455411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:54.460268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34747337-14e0-46da-867e-33a5503b5f63 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:54.460691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34747337-14e0-46da-867e-33a5503b5f63 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:54.460794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34747337-14e0-46da-867e-33a5503b5f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:54.516107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:54.538914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-34747337-14e0-46da-867e-33a5503b5f63 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:54.540139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-34747337-14e0-46da-867e-33a5503b5f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:54.640228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:54.796835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:54.798296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:54.950071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9d4d1-e988-4ec6-81ca-d5ef4aac709f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:54.962297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:55.064321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:55.064843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:55.065123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:55.065465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:55.067028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:55.158735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:55.160364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6beb3-3267-40c0-a8b9-4b2eff2a1581 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:55.178568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 716/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:56.061889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:56.062426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:56.062426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:56.063804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:43:56.069072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f7bc-1cee-4ee5-861c-dc9dd8758b09 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:56.069517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f7bc-1cee-4ee5-861c-dc9dd8758b09 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:56.069631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98f7bc-1cee-4ee5-861c-dc9dd8758b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:56.090123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:56.147222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6f98f7bc-1cee-4ee5-861c-dc9dd8758b09 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:56.148438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6f98f7bc-1cee-4ee5-861c-dc9dd8758b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:56.247238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:56.400140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:56.402909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:56.523336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:56.560881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a757-f0c2-4b83-b9da-55738492766d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:56.680407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:56.681009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:56.681278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:56.681541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:56.684141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:56.761369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:56.775614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:56.777798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a8c29-4b96-45d9-af7d-20e43975dc89 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:56.796633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 708/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:57.661012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:57.661575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:57.661575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:57.662903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:57.667805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892e9f-e228-4ef9-8458-429bedab200a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:57.668269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892e9f-e228-4ef9-8458-429bedab200a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:57.668368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892e9f-e228-4ef9-8458-429bedab200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:57.747553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1e892e9f-e228-4ef9-8458-429bedab200a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:57.748799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1e892e9f-e228-4ef9-8458-429bedab200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:57.859095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:43:58.012342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:58.014211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:58.107991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:58.174315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5b2e8-f54b-4021-9f07-4ee41392c7c1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:58.289733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:58.290285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:58.290566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:58.290838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:58.292443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:58.342523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:58.384755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:58.386985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5da46-f92a-43d7-ab30-9fa039188cb0 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:58.405145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 719/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:43:59.229257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:59.230151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:59.230151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:59.231248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:43:59.236353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aee51-3fd0-4cd5-a14b-f5906e7ab604 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:59.236776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aee51-3fd0-4cd5-a14b-f5906e7ab604 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:59.236898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aee51-3fd0-4cd5-a14b-f5906e7ab604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:59.315124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fa7aee51-3fd0-4cd5-a14b-f5906e7ab604 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:43:59.316334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fa7aee51-3fd0-4cd5-a14b-f5906e7ab604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:59.413873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:43:59.565899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:43:59.567433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:59.616095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:59.721979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa851d-bffe-46b4-a56e-ab69d9af6f6b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:43:59.839123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:43:59.839717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:43:59.839988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:43:59.840250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:43:59.841852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:43:59.848651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:43:59.932038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:43:59.933668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a5362-ce7e-4036-9e33-b2e92cb37f3b service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:43:59.952203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 711/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:00.773403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:00.774173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:00.774173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:00.775347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:00.781454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509b63-25fb-40c8-b35a-8aa7f9f0a32e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:00.781909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509b63-25fb-40c8-b35a-8aa7f9f0a32e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:00.782042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509b63-25fb-40c8-b35a-8aa7f9f0a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:00.880102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-30509b63-25fb-40c8-b35a-8aa7f9f0a32e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:00.881327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-30509b63-25fb-40c8-b35a-8aa7f9f0a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:00.951191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:00.982033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:01.137529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:01.139876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:01.169442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:01.313048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea1846-5398-45ec-b1ae-a70c750236ee None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:01.409661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:01.438158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:01.438781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:01.439058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:01.439330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:01.441016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:01.550292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:01.551922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e4b28-2c16-4699-be72-362668f88e12 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:01.570873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 722/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:02.416411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:02.417003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:02.417003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:02.418398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:44:02.424024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a47ca-fc58-4abe-a1f0-3c32cbcb1b54 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:02.424544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a47ca-fc58-4abe-a1f0-3c32cbcb1b54 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:02.424642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a47ca-fc58-4abe-a1f0-3c32cbcb1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:02.506339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e64a47ca-fc58-4abe-a1f0-3c32cbcb1b54 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:02.507549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e64a47ca-fc58-4abe-a1f0-3c32cbcb1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:02.552024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:02.617067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:02.769853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:02.771340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:02.971276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce963b7b-308d-4ef0-943a-d5ce7721ac32 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:03.031333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:03.094834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:03.095445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:03.095721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:03.095994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:03.097607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:03.188344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:03.189992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79f98b-c104-49ab-8f5b-80f55cf2e27e service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:03.209019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 714/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:04.032502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:04.033057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:04.033057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:04.034464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:04.043692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97bc97-71fa-4844-abb3-91c759aefdb5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:04.044278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97bc97-71fa-4844-abb3-91c759aefdb5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:04.044278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97bc97-71fa-4844-abb3-91c759aefdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:04.133884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6b97bc97-71fa-4844-abb3-91c759aefdb5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:04.135146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6b97bc97-71fa-4844-abb3-91c759aefdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:04.137440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:04.240568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:04.397628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:04.399382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:04.563658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48395-71dd-4d23-8560-efce2bc30a3a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:04.595646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:04.696347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:04.696967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:04.697253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:04.697526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:04.699778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:04.800757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:04.802406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b1762-cb30-48ac-80af-24824396f27c service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:04.827446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 725/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:06.683864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:06.684456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:06.684456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:06.685862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:06.691781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3f90-4f01-4067-82bd-3cf6d013a980 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:06.692348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3f90-4f01-4067-82bd-3cf6d013a980 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:06.692348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c3f90-4f01-4067-82bd-3cf6d013a980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:06.737900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:06.782385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c96c3f90-4f01-4067-82bd-3cf6d013a980 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:06.783617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c96c3f90-4f01-4067-82bd-3cf6d013a980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:06.907451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:07.229452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:07.231951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:07.368967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91cca9-8e42-4dfb-a182-bd403fcd4138 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:07.409876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:07.464360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:07.464973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:07.465247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:07.465519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:07.467306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:07.613718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:07.615298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ade2f2-0c0b-4c1b-805a-fd0d168f24a3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:07.637485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 717/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:09.817209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:09.817890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:09.817890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:09.819137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:09.825349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cb4a-7861-453e-9887-b226d5640dca None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:09.825908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cb4a-7861-453e-9887-b226d5640dca None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:09.826013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111cb4a-7861-453e-9887-b226d5640dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:09.897510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5111cb4a-7861-453e-9887-b226d5640dca None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:09.898806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5111cb4a-7861-453e-9887-b226d5640dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:09.912405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:10.007148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:10.298550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:10.300137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:10.419448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664120c-cdef-4ff2-b3d4-bc2a470cc387 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:10.434075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:10.505313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:10.505878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:10.506157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:10.506429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:10.508091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:10.620122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:10.621745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf68db-1f54-4345-942f-51f62ceec032 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:10.641302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 728/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:12.891195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:12.891787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:12.891787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:12.893276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:12.898355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c184a-bb72-4bf2-ba9d-487cb2a61b84 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:12.898782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c184a-bb72-4bf2-ba9d-487cb2a61b84 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:12.898912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c184a-bb72-4bf2-ba9d-487cb2a61b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:12.949186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:12.968398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2c5c184a-bb72-4bf2-ba9d-487cb2a61b84 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:12.969613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2c5c184a-bb72-4bf2-ba9d-487cb2a61b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:13.068433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:13.359530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:13.361076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:13.466746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4b135-9790-4dce-91be-df2038ee5219 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:13.506439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:13.554263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:13.554890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:13.555169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:13.555443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:13.557811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:13.670118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:13.671570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b884b6-3dca-49dd-9941-bc78ce8d8ab1 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:13.690653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 720/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:15.841528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:15.842186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:15.842186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:15.843517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:15.849388 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d50c-7090-47ad-a606-57281ed02e35 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:15.850012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d50c-7090-47ad-a606-57281ed02e35 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:15.850012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d50c-7090-47ad-a606-57281ed02e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:15.920057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f801d50c-7090-47ad-a606-57281ed02e35 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:15.921287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f801d50c-7090-47ad-a606-57281ed02e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:15.942589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:16.021626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:16.319529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:16.321070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:16.426005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f271d8-017d-4f06-94cc-cd559cce4535 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:16.448699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:16.512368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:16.512883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:16.513154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:16.513419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:16.515306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:16.626326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:16.627776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc189c-e3dc-4518-8db0-6fcd17572583 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:16.647500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 731/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:18.646656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:18.647244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:18.647244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:18.648402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:18.654681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9ce5-2dfc-4050-a35d-b50faf75c2ca None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:18.655234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9ce5-2dfc-4050-a35d-b50faf75c2ca None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:18.655234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9ce5-2dfc-4050-a35d-b50faf75c2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:18.721967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:18.725207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-229e9ce5-2dfc-4050-a35d-b50faf75c2ca None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:18.727084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-229e9ce5-2dfc-4050-a35d-b50faf75c2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:18.825951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:19.108585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:19.110159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:19.216193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df15450-5194-43da-97e6-0c98a9cda91a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:19.267398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:19.293250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:19.293888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:19.294160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:19.294432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:19.296022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:19.408744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:19.410128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44e2cd-2228-4842-96bf-e455ad9b6c4f service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:19.428693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 723/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:21.440976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:21.441550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:21.441550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:21.443048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:21.448841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365f16-b050-4c1d-a79a-d26eff629581 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:21.449282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365f16-b050-4c1d-a79a-d26eff629581 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:21.449384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365f16-b050-4c1d-a79a-d26eff629581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:21.519283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b4365f16-b050-4c1d-a79a-d26eff629581 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:21.520496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b4365f16-b050-4c1d-a79a-d26eff629581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:21.550613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:21.621226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:21.907775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:21.909320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:22.013167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ea09b-43f7-4cc6-b1e5-24f322a5e1a3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:22.042972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:22.091641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:22.092255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:22.092534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:22.092804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:22.094442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:22.209574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:22.211167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bead0fe-37aa-4dcb-abd6-987d178a8017 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:22.229537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 734/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:24.203143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:24.203725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:24.203725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:24.205079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:24.210423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b80b14-b621-44a2-8ba7-5ddac7b437e8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:24.210880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b80b14-b621-44a2-8ba7-5ddac7b437e8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:24.211354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b80b14-b621-44a2-8ba7-5ddac7b437e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:24.280541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-20b80b14-b621-44a2-8ba7-5ddac7b437e8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:24.281773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-20b80b14-b621-44a2-8ba7-5ddac7b437e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:24.289864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:24.381312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:24.667924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:24.669492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:24.777606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c73d-4d5c-464d-9ca1-2d8fbd463bc6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:24.836106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:24.855614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:24.856216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:24.856490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:24.856756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:24.858381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:24.971020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:24.972365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83d70-9bf4-495a-b2d3-0771cf0ed9b2 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:24.991323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 726/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:27.044135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:27.044713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:27.044713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:27.046059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:27.051307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97d6d-bb77-4161-98ca-447c7cdb0987 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:27.051721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97d6d-bb77-4161-98ca-447c7cdb0987 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:27.051856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97d6d-bb77-4161-98ca-447c7cdb0987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:27.117979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e3b97d6d-bb77-4161-98ca-447c7cdb0987 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:27.119704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e3b97d6d-bb77-4161-98ca-447c7cdb0987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:27.167793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:27.227393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:27.514622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:27.516185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:27.745400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19438f4-44a0-4eb4-a875-89c4936ea7ae None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:27.784599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:27.827722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:27.828359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:27.828633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:27.828898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:27.830608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:27.941216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:27.942670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61481d06-7fe4-44c5-851a-5328a31fd680 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:27.960881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 737/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:29.936568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:29.937117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:29.937117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:29.938517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:29.943996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb16f3-5447-4ab9-a94d-0d1fbb7d508b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:29.944460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb16f3-5447-4ab9-a94d-0d1fbb7d508b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:29.944557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb16f3-5447-4ab9-a94d-0d1fbb7d508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:30.011118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-81eb16f3-5447-4ab9-a94d-0d1fbb7d508b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:30.012337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-81eb16f3-5447-4ab9-a94d-0d1fbb7d508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:30.031158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:30.108601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:30.397476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:30.399030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:30.502733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41984c-d4b8-44ad-b0dc-f087419ed130 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:30.569845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:30.577484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:30.578105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:30.578380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:30.578652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:30.580226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:30.708593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:30.710120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010baf8b-36ea-41fc-bcd5-92654a587b1c service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:30.728149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 729/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:32.658857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:32.659395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:32.659395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:32.660727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:32.666153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866adfff-5634-4191-b6cb-194173d1adcc None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:32.666546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866adfff-5634-4191-b6cb-194173d1adcc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:32.666729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866adfff-5634-4191-b6cb-194173d1adcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:32.733883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-866adfff-5634-4191-b6cb-194173d1adcc None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:32.735084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-866adfff-5634-4191-b6cb-194173d1adcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:32.783830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:32.835348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:33.123509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:33.125071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:33.230942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecc8a7-4763-4796-843d-9410f24dbd49 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:33.279777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:33.306057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:33.306714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:33.306984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:33.307249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:33.308811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:33.418438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:33.419864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b326b-000a-464e-b5f7-fde7e9ca3d75 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:33.437768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 740/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:42.833229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=22 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:43.435345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=66 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:53.446167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311a72-16d7-40f9-9bae-0a2b8b5d9603 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:53.446803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311a72-16d7-40f9-9bae-0a2b8b5d9603 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:53.446803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef311a72-16d7-40f9-9bae-0a2b8b5d9603 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:53.448088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 731/1471] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:53.453949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42c61e-e0bd-4261-9d10-f576676b6c8e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:53.454404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42c61e-e0bd-4261-9d10-f576676b6c8e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:53.454508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42c61e-e0bd-4261-9d10-f576676b6c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:53.522261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-df42c61e-e0bd-4261-9d10-f576676b6c8e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:53.523477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-df42c61e-e0bd-4261-9d10-f576676b6c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:53.619672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 741/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:53.921425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:53.921764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:53.922042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:53.923383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 732/1473] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 6 11:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:53.928311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758a5d-81e2-4656-a2f8-e65cab8f3b21 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:53.928601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758a5d-81e2-4656-a2f8-e65cab8f3b21 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:53.928876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758a5d-81e2-4656-a2f8-e65cab8f3b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:54.006731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a6758a5d-81e2-4656-a2f8-e65cab8f3b21 None None] MFA Rules not processed for user `7739336ec72248feb11d1d6a7a8de166`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:54.007960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a6758a5d-81e2-4656-a2f8-e65cab8f3b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:54.031588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:54.090582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 742/1474] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 6 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:54.098241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:54.099767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:54.207718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48dd0-5b94-474c-8600-9ac266a9dba7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:54.284292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:54.284897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:54.285170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:54.285439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:54.287118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:54.346365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:54.406452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:54.407988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1d766-3a64-42dd-b61c-8903c021ab4f service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:54.427176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 733/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:44:56.507449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:56.508018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:56.508018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:56.509366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 743/1476] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:56.515158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ef6-25be-4b27-b156-5c8a9e2267a3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:56.515599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ef6-25be-4b27-b156-5c8a9e2267a3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:56.515695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ef6-25be-4b27-b156-5c8a9e2267a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:56.586142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-80a29ef6-25be-4b27-b156-5c8a9e2267a3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:56.587366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-80a29ef6-25be-4b27-b156-5c8a9e2267a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:56.687573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 734/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:56.963561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:56.965086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:57.094609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480849a2-572a-4d80-8a45-ad4d1ac02614 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:44:57.146198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:57.198964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:44:57.199588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:57.199868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:57.200142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:57.200788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:44:57.202594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:44:57.202594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:44:57.202594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:44:57.203760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78241fa4-45ce-4e23-bd54-e2cb2f447866 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:44:57.224657 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 744/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 6 11:44:56 2022] GET /identity/v3/projects => generated 2553 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:44:59.299234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:59.299807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:59.299807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:59.301186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 735/1479] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:44:59.307409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b19d09-d21f-4f80-aa7b-06a61acea32d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:44:59.307837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b19d09-d21f-4f80-aa7b-06a61acea32d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:44:59.307949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b19d09-d21f-4f80-aa7b-06a61acea32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:44:59.379056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-19b19d09-d21f-4f80-aa7b-06a61acea32d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:44:59.380265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-19b19d09-d21f-4f80-aa7b-06a61acea32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:59.477231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 745/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:44:59.780891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:44:59.782535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:44:59.793013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:44:59.934357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfcbf1-528b-416d-9c99-da0c0c6a4d32 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:45:00.014350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:00.048574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:45:00.049104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:00.049381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:00.049678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:00.051337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:00.140729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:45:00.142404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69ce6e-135c-47a1-a779-5c85467c8d32 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:45:00.161135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 736/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:45:02.329584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:02.330204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:02.330204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:02.331514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 746/1482] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:45:02.336941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af12f6-156e-444b-95fe-44b7c1bda06a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:02.337768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af12f6-156e-444b-95fe-44b7c1bda06a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:02.337768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af12f6-156e-444b-95fe-44b7c1bda06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:02.412163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-80af12f6-156e-444b-95fe-44b7c1bda06a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:45:02.413424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-80af12f6-156e-444b-95fe-44b7c1bda06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:02.466744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:02.522328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 737/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:45:02.841723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:45:02.843826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:02.872262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:02.964356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f14e2-6f6c-4645-986d-7cb4a14bc616 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:45:03.045521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:45:03.046167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:03.046438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:03.046708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:03.048356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:03.133234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:03.172743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:45:03.174244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a3f10-11c1-4f56-adac-109ead10ee79 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:45:03.193404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 747/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:45:05.185029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:05.185537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:05.185632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:05.187013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 738/1485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:45:05.192240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709059e-0f45-45bb-a015-db73cd1ea6ed None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:05.192670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709059e-0f45-45bb-a015-db73cd1ea6ed None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:05.192804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709059e-0f45-45bb-a015-db73cd1ea6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:05.261782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6709059e-0f45-45bb-a015-db73cd1ea6ed None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:45:05.263013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6709059e-0f45-45bb-a015-db73cd1ea6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:05.378526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 748/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:45:05.676351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:45:05.677930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:05.789886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603eedb-c2b8-4366-84f0-33870f7c492d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:45:05.864346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:05.866278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:45:05.866897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:05.867374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:05.867864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:05.870695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:05.981415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:45:05.982837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdba2e-6b99-451f-ac90-46e37aefe49e service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:45:06.001439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 739/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:45:08.101112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:08.101830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:08.101830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:08.103110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 749/1488] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:45:08.108558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30372-0a8c-4ea8-b328-6e837914befd None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:08.109012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30372-0a8c-4ea8-b328-6e837914befd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:08.109198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30372-0a8c-4ea8-b328-6e837914befd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:08.182646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3aa30372-0a8c-4ea8-b328-6e837914befd None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:45:08.183778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3aa30372-0a8c-4ea8-b328-6e837914befd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:08.242000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:08.293729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 740/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:45:08.588964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:45:08.590624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:08.628407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:08.701528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eaadd-7999-4d0a-9d26-e7574fb0ceb4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:45:08.782656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:45:08.783270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:08.783548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:08.783818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:08.785546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:08.873534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:08.907526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:45:08.910116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195ea79-1349-4d6e-bca3-59979a2fdf62 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:45:08.933534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 750/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:45:11.146144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:11.146757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:11.146757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:11.148036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 741/1491] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 6 11:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:45:11.154285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa58e2-c14c-4135-936f-298575ba10df None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:11.154713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa58e2-c14c-4135-936f-298575ba10df None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:11.154958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa58e2-c14c-4135-936f-298575ba10df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:11.220875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-cdfa58e2-c14c-4135-936f-298575ba10df None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:45:11.222125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cdfa58e2-c14c-4135-936f-298575ba10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:11.320395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 751/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 6 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:45:11.623709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:45:11.625268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:11.731190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1ee7-c3da-4d6f-b1ca-be8af2615b9d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:45:11.806719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:45:11.807315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:45:11.807594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:45:11.807868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:45:11.809475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:45:11.818782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:11.921525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:45:11.923007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46d2d-cb52-4a2e-9ee2-3961e31b05b3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:45:11.942252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 742/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:45:21.318638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=82 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:45:21.939899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=50 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:25.722987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:25.723599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:25.723599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:25.805788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:46:25.807032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:25.914496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 752/1494] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:46:25.923026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:25.923582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:25.923582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:26.011026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:46:26.012183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:26.122920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 743/1495] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:46:26.127793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:26.129064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:26.141389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c043a92-4632-4e80-8d7c-7055a0f14a67 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:26.214414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:26.254705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:26.255323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:26.255600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:26.255871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:26.256520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:46:26.256823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:26.257930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:26.257930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:46:26.257930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:46:26.258844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:26.271820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 753/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:46:26.280465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:26.292849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb74831-cc74-4fd4-95ba-c3d2eb4f1e01 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:26.362708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:26.401148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:26.401835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:26.402117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:26.402414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:26.403068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:26.404923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:26.404923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:46:26.404923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:46:26.406214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:26.469692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 744/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:46:26 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:46:26.474075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:26.475749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:26.514841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f216-0ede-457a-9736-60b90d0b23bb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:26.623498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:26.624029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:26.624306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:26.624573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:26.625232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:26.626375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:26.626375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:46:26.626375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:46:26.627618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:26.678811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 754/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:46:26 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:46:26.685301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:26.687202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:26.756217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eace91f-4a69-4c12-a422-b47b809400eb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:26.849111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:26.871214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:26.871823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:26.872098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:26.872369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:26.873013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:26.874111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:26.874111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:46:26.874111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:46:26.874965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:26.888581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 745/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:46:26.894556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:26.896064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:26.908398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f578741-9d3d-4783-9b6f-55f631168bcd admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:26.934405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:27.010408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:27.011017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.011286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.011550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.012192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:27.013200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:27.013200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:46:27.013200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:46:27.014115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dca820-2598-428b-81f7-75af489d975f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:27.027530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 755/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:46:27.032787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:27.034419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:27.068612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88d34-b122-4d48-b02d-56a0bea6c9a0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:27.170250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:27.170920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.171192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.171459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] PATH_INFO: `/v3/projects/2a83e78d639a4731ae670171353c9adb/users/e4cce3cb0fbf43d2b6672e5ffe902e3a/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.195570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a83e78d639a4731ae670171353c9adb, user_id=e4cce3cb0fbf43d2b6672e5ffe902e3a, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:27.196616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:27.196616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:46:27.196616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:46:27.198596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4d3c6-92a7-421e-8c0f-382fe7b36db3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:27.224256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 746/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:46:27 2022] PUT /identity/v3/projects/2a83e78d639a4731ae670171353c9adb/users/e4cce3cb0fbf43d2b6672e5ffe902e3a/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:46:27.237162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6b91-c914-4eed-b9fa-f0bf441a5e0c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.237463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6b91-c914-4eed-b9fa-f0bf441a5e0c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.237747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6b91-c914-4eed-b9fa-f0bf441a5e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.299372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7fcb6b91-c914-4eed-b9fa-f0bf441a5e0c None None] MFA Rules not processed for user `e4cce3cb0fbf43d2b6672e5ffe902e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:46:27.300587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7fcb6b91-c914-4eed-b9fa-f0bf441a5e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:27.384398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 756/1502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:46:27.398341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.398341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.399081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.400260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 747/1503] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 6 11:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:46:27.405357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a53e9-024f-4e71-b7e9-081b1510a27e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.405663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a53e9-024f-4e71-b7e9-081b1510a27e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.405955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a53e9-024f-4e71-b7e9-081b1510a27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.440232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:27.488281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fd1a53e9-024f-4e71-b7e9-081b1510a27e None None] MFA Rules not processed for user `ccafefb49d0c4727bad188927ae5bf7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:46:27.489544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fd1a53e9-024f-4e71-b7e9-081b1510a27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:27.587133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 757/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 6 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:46:27.595305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:27.596887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:27.653301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:27.710896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b1dc1-68df-4670-a8db-3da46b91e040 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:27.789009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:27.789628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.789930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.790199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.791850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:27.908064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:27.909532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:27.913358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:27.931540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 748/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:46:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:46:27.979451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcb0e-1105-4c6a-ba0e-e53875e71ca0 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:27.979941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcb0e-1105-4c6a-ba0e-e53875e71ca0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:27.980051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcb0e-1105-4c6a-ba0e-e53875e71ca0 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:27.981264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 758/1506] 10.222.0.18 () {52 vars in 830 bytes} [Tue Sep 6 11:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:46:27.985095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:27.986364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:28.214644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511df3-887e-4633-8917-9f216d51cfdf service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:28.313188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:28.313839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:28.314114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:28.314379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] PATH_INFO: `/v3/users/e4cce3cb0fbf43d2b6672e5ffe902e3a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:28.328327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4cce3cb0fbf43d2b6672e5ffe902e3a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:28.329366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:28.329366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:46:28.329366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:46:28.330608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:28.341207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:28.413027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4cce3cb0fbf43d2b6672e5ffe902e3a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:28.413862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4cce3cb0fbf43d2b6672e5ffe902e3a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:28.439298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4cce3cb0fbf43d2b6672e5ffe902e3a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:28.449047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4cce3cb0fbf43d2b6672e5ffe902e3a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:28.449844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4cce3cb0fbf43d2b6672e5ffe902e3a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:28.470007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 749/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:46:27 2022] DELETE /identity/v3/users/e4cce3cb0fbf43d2b6672e5ffe902e3a => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:46:28.474440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:28.474729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:28.475007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:28.541511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:46:28.542888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:28.624445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:28.640455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 759/1508] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:46:28.650752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:28.652009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:28.763250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73a95-1b7a-4b4a-a3ac-bad473faccd4 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:28.829482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:28.864618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:28.865696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:28.866185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:28.867152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:28.870491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:29.030504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:29.031985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de6d4d-2f55-4dc5-8fdc-22699ad52ee0 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:29.052950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 750/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:46:29.508087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:46:29.509343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:46:29.523338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0352-a830-4b54-a144-3625e32d2bf5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:46:29.639435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:46:29.640350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:46:29.640629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:46:29.640899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] PATH_INFO: `/v3/projects/2a83e78d639a4731ae670171353c9adb` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:46:29.657969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a83e78d639a4731ae670171353c9adb)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:46:29.659253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:46:29.659253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:46:29.659253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:46:29.660753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:46:29.729480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a83e78d639a4731ae670171353c9adb', 'request_id': 'req-e2f843ac-2ba0-4525-81c9-277bb1acb31a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:29.737426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a83e78d639a4731ae670171353c9adb', 'request_id': 'req-e2f843ac-2ba0-4525-81c9-277bb1acb31a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:29.750313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a83e78d639a4731ae670171353c9adb', 'request_id': 'req-e2f843ac-2ba0-4525-81c9-277bb1acb31a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:29.784862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] The token cache is being invalidate because project 2a83e78d639a4731ae670171353c9adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:46:29.785155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:29.785735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e2f843ac-2ba0-4525-81c9-277bb1acb31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:46:29.787088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 760/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:46:29 2022] DELETE /identity/v3/projects/2a83e78d639a4731ae670171353c9adb => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:46:39.050249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=62 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:46:39.782624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:48:01.368502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:01.369102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:01.369102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:01.370524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:01.375637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be114d57-c36c-4316-8907-684205cc26c9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:01.376093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be114d57-c36c-4316-8907-684205cc26c9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:01.376196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be114d57-c36c-4316-8907-684205cc26c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:01.461072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-be114d57-c36c-4316-8907-684205cc26c9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:01.462848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-be114d57-c36c-4316-8907-684205cc26c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:01.569348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:01.723085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:01.725279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:01.849014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea882c3b-2af5-437d-a43f-a7476e328398 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:01.855214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:01.855819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:01.856102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:01.856378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:01.864690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:01.866336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc9e50-2b47-4689-8e57-1fe3b45fff6c service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:01.876518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 752/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:02.720193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd70-a187-4b9b-8eb3-7e0416949710 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:02.720794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd70-a187-4b9b-8eb3-7e0416949710 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:02.720794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd70-a187-4b9b-8eb3-7e0416949710 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:02.722281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:02.727889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7b5b-6ea9-4613-8d66-e6a5b253c598 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:02.728526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7b5b-6ea9-4613-8d66-e6a5b253c598 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:02.728526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7b5b-6ea9-4613-8d66-e6a5b253c598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:02.749977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b57c7b5b-6ea9-4613-8d66-e6a5b253c598 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:02.751209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b57c7b5b-6ea9-4613-8d66-e6a5b253c598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:02.759371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:02.917967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbd70-a187-4b9b-8eb3-7e0416949710 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:02.932321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbd70-a187-4b9b-8eb3-7e0416949710 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:02.938067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:02.938676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:02.938958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:02.939228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:02.948317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:02.949970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7213b-d469-4e0a-b52c-e8d4b322f7a6 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:02.959404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 763/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:04.113083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec7776-0cef-43b8-a1fb-b124c54a8f1f None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:04.113625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec7776-0cef-43b8-a1fb-b124c54a8f1f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:04.113625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec7776-0cef-43b8-a1fb-b124c54a8f1f None None] PATH_INFO: `/v3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:04.114960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 754/1517] 10.222.0.64 () {54 vars in 925 bytes} [Tue Sep 6 11:48:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:48:04.121767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9f97-6bf4-4980-b062-708975b03e75 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:04.122238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9f97-6bf4-4980-b062-708975b03e75 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:04.122353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9f97-6bf4-4980-b062-708975b03e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:04.182795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-536f9f97-6bf4-4980-b062-708975b03e75 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:04.183996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-536f9f97-6bf4-4980-b062-708975b03e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:04.201362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:48:04.258062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 764/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Sep 6 11:48:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:12.744448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=56 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:48:14.236915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d942-7f0e-421a-85ab-de9bde6bc8ce None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.237607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d942-7f0e-421a-85ab-de9bde6bc8ce None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.237607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d942-7f0e-421a-85ab-de9bde6bc8ce None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.238848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 755/1519] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:14.244140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e41-dc76-4665-af32-cd84fb5633a6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.244656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e41-dc76-4665-af32-cd84fb5633a6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.244656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e41-dc76-4665-af32-cd84fb5633a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.267661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c4f57e41-dc76-4665-af32-cd84fb5633a6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:14.268889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c4f57e41-dc76-4665-af32-cd84fb5633a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:14.278552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 765/1520] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:14.409538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f218-0c2f-4549-a560-d84ecb1d47c0 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.410053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f218-0c2f-4549-a560-d84ecb1d47c0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.410398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f218-0c2f-4549-a560-d84ecb1d47c0 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.411788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 756/1521] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 6 11:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:14.417870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.418172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.418450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.441718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:14.443142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:14.452772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 766/1522] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 6 11:48:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:14.460427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f218-0c2f-4549-a560-d84ecb1d47c0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:14.462512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:14.475264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652aa1d2-c89a-4a74-8e41-459f2ccc0194 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:14.478940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f218-0c2f-4549-a560-d84ecb1d47c0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:14.482024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:14.482645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.482924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.483220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.484803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:14.485423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:14.485731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:14.486005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:14.494032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:14.495775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587dc7a-7eb9-4a96-a1ff-02c5cea46c09 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:14.496433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:14.498178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743be9e-9c64-491d-af92-9d88be1e05ee service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:14.507021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 767/1523] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:14.510582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 757/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:15.349961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809af2-0267-472b-81fb-fc32638118ac None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:15.350500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809af2-0267-472b-81fb-fc32638118ac None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:15.350500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809af2-0267-472b-81fb-fc32638118ac None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:15.352162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 11:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:15.357351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf41-24ab-42c5-9675-3b2004d8ddc4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:15.357866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf41-24ab-42c5-9675-3b2004d8ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:15.357971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf41-24ab-42c5-9675-3b2004d8ddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:15.380248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9ad6cf41-24ab-42c5-9675-3b2004d8ddc4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:15.381527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9ad6cf41-24ab-42c5-9675-3b2004d8ddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:15.392451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 11:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:15.552883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809af2-0267-472b-81fb-fc32638118ac None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:15.574605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809af2-0267-472b-81fb-fc32638118ac None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:15.580604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:15.581239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:15.581519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:15.581821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:15.591539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:15.593155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f969-716d-4ade-a6cb-5b0847e91e6d service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:15.603553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 769/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:19.433864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:19.434434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:19.434434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:19.435836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:19.441232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede801c6-6e05-472d-802b-1be0ef898829 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:19.441726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede801c6-6e05-472d-802b-1be0ef898829 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:19.441726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede801c6-6e05-472d-802b-1be0ef898829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:19.463305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ede801c6-6e05-472d-802b-1be0ef898829 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:19.464512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ede801c6-6e05-472d-802b-1be0ef898829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:19.472537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:19.763532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:19.765235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:19.845965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec080f-d822-4abd-9e5e-e951e02b4a39 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:19.849680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:19.850286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:19.850564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:19.850833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:19.859947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:19.861384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c933a-8b7a-45a6-b9ee-bf1c403a0a97 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:19.871238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 760/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:21.813245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06413-f093-4fe2-bf78-ce633002ef64 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:21.813821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06413-f093-4fe2-bf78-ce633002ef64 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:21.813821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06413-f093-4fe2-bf78-ce633002ef64 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:21.815164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:21.820486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c9c8-fb3e-4d79-8ba8-fff1656006a3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:21.821007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c9c8-fb3e-4d79-8ba8-fff1656006a3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:21.821007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c9c8-fb3e-4d79-8ba8-fff1656006a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:21.841103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ae56c9c8-fb3e-4d79-8ba8-fff1656006a3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:21.842361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ae56c9c8-fb3e-4d79-8ba8-fff1656006a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:21.850269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:22.142698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b06413-f093-4fe2-bf78-ce633002ef64 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:22.155566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b06413-f093-4fe2-bf78-ce633002ef64 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:22.159380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:22.159996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:22.160267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:22.160537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:22.170909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:22.172331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a98ca-470e-40d0-8069-5c8eb098e7d1 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:22.182035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 772/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:25.667999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:25.668566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:25.668566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:25.669935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:25.675213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221ee9-9f5f-4d32-9766-f7250793292d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:25.675612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221ee9-9f5f-4d32-9766-f7250793292d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:25.675762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8221ee9-9f5f-4d32-9766-f7250793292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:25.695917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d8221ee9-9f5f-4d32-9766-f7250793292d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:25.697163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d8221ee9-9f5f-4d32-9766-f7250793292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:25.705156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:26.273506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:26.276215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:26.357469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880d6ec-16ff-431c-bb66-22805ac9ed1b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:26.360999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:26.361598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:26.361911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:26.362171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:26.372007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:26.374179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bacb2-3c17-4295-a475-97482d042852 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:26.386767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 763/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:28.355844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:28.356376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:28.356376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:28.357674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 11:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:48:28.362822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeff9-9c5a-47de-8fb2-336983267890 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:28.363284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeff9-9c5a-47de-8fb2-336983267890 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:28.363390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeff9-9c5a-47de-8fb2-336983267890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:28.383290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-80faeff9-9c5a-47de-8fb2-336983267890 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:48:28.384650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-80faeff9-9c5a-47de-8fb2-336983267890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:28.392624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 11:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:48:28.956873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:48:28.958666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:48:28.971724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa1d49-1808-4941-aa44-d76c4dc3dcb5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:48:28.975253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:48:28.975857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:48:28.976131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:48:28.976401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:48:28.986037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:48:28.987452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dfca5-aac9-405b-abe7-3f509a6c7956 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:48:28.997459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 775/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:48:38.378375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=85 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:48:38.992665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=59 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:54.866294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba107ed-06e5-4b24-ae00-a423003c795c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:54.868256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba107ed-06e5-4b24-ae00-a423003c795c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:54.868895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba107ed-06e5-4b24-ae00-a423003c795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:54.909819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00613b6-04fa-4962-86e1-769c5be554cd None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:54.910763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00613b6-04fa-4962-86e1-769c5be554cd None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:54.910763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00613b6-04fa-4962-86e1-769c5be554cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:54.916268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-7ba107ed-06e5-4b24-ae00-a423003c795c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:54.918954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7ba107ed-06e5-4b24-ae00-a423003c795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:54.942514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 765/1540] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:54.960343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:54.962773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f00613b6-04fa-4962-86e1-769c5be554cd None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:54.965159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f00613b6-04fa-4962-86e1-769c5be554cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:54.966075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:54.966869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:54.996538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 776/1541] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:49:55.017187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.022269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.023125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.026295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:55.027528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:55.048385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 766/1542] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:55.073060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.075341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:55.077912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:55.093020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4942c-3ef0-4942-9e91-18d1d646ad0c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.097689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 777/1543] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:55.106430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.114352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.115075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.115254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.115541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.116974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:55.117259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.118266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:49:55.118266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:49:55.118266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:49:55.119161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.132944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe0d4-a1d1-47f5-9786-45a9676c8fdd None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.144830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.146353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.148334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.149135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.150826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:55.151014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 767/1544] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:55.151850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.155299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.191731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 778/1545] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:55.206022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.209073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.232858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda71a60-53d5-450d-b759-ea975ef7364e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.240130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e9d28-3773-43eb-9c06-0b41f56b5d77 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.250145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.251826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.252029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.252862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.253329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.261428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.261816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.261816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.261816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.263013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.265164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.273697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:49:55.273697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:49:55.273697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:49:55.273697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.361038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 779/1546] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:55.363977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 768/1547] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:55.371016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.371944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.498470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d0fee-3f74-4d24-a9b2-868db46709db admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.511896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.512669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.513051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.513427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.514327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.515725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:49:55.515725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:49:55.515725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:49:55.517243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.534973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63863-30e9-4e61-9c95-f5a586afc036 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.545992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.546660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.546932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.547233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.547902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.550295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.560952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 769/1548] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:49:55.566691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.591171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1720a58-5441-4db8-934c-3ed1af8c5079 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.607889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.608503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.609049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.609330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.610011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.611636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.617768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 780/1549] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:49:55.624154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.640411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 770/1550] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:55.644861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529ff23-1d31-4305-b56c-9431bb0d844d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.646580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.664705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.665801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.666329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.666806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.667957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.668887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19342e0f-944c-4f13-aba3-457381a3c715 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.677036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.690930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.691568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.691847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.692123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.692781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.695437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.703847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 781/1551] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:55.711641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.721412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 771/1552] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:49:55.727673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.744960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f743e-be7c-4dff-b782-1cc3156c10ea admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.745668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b5709-5763-4681-9381-19f4670b8568 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.759811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.760999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.761293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.761556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] PATH_INFO: `/v3/projects/a896d72b5b00477d855131345711c748/users/e235718ad1da42ebb209fb4b13626ff1/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.764007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.765178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.765968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.766437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.767790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.774312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.802689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a896d72b5b00477d855131345711c748, user_id=e235718ad1da42ebb209fb4b13626ff1, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:55.806709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec573e06-b37e-4a2d-8188-425e2dbd3b80 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:55.809396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 782/1553] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:49:55.817632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:55.836104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 772/1554] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:55 2022] PUT /identity/v3/projects/a896d72b5b00477d855131345711c748/users/e235718ad1da42ebb209fb4b13626ff1/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:49:55.842618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.842911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.843198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:55.845182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec9504-a1aa-4920-aba3-1cce0e9e9008 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:55.878779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:55.880332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:55.973401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:55.987750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:55.988493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:55.988694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:55.988964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] PATH_INFO: `/v3/projects/83ee0f5bbd6a485991984fe0bf4ffd44/users/739ec5079888451d93c4bcd38812d54b/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.024512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 773/1555] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:49:56.041675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ee0f5bbd6a485991984fe0bf4ffd44, user_id=739ec5079888451d93c4bcd38812d54b, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:56.041675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:49:56.041675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:49:56.041675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:49:56.041675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a492540-06ea-4261-8d6e-a699f11300d7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:56.056570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:56.088137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 783/1556] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:55 2022] PUT /identity/v3/projects/83ee0f5bbd6a485991984fe0bf4ffd44/users/739ec5079888451d93c4bcd38812d54b/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:56.132540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e214e-cb71-4344-ae0b-e4e590feb283 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.132843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e214e-cb71-4344-ae0b-e4e590feb283 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.133090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e214e-cb71-4344-ae0b-e4e590feb283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.150562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:56.188838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f57e214e-cb71-4344-ae0b-e4e590feb283 None None] MFA Rules not processed for user `739ec5079888451d93c4bcd38812d54b`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:56.190077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f57e214e-cb71-4344-ae0b-e4e590feb283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:56.208349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4352eec-d6ad-4e4a-bde8-2bc7e44efa04 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:56.325707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 784/1557] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:56.326219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.326504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.326785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.378826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:56.380254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.380968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.381837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.399041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:56.400175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:56.552992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:56.555652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f84d31-335e-4b3f-821b-663a78c83d65 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:56.559233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 785/1558] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:56.560123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:56.583527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e488cf7-fe1f-49fc-af0e-cc49f851693b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:56.586869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 774/1559] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:56.597700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.597700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.597700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.599242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:56.599837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.600137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.600423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.601050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:56.605704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5a829-711f-48b8-9fe9-e96932bbf8bd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:56.641090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:56.648685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:56.669527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 786/1560] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:56.671164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.671457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.671756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.679847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 775/1561] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:56.679847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:56.713678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7dbad-06ef-4fe0-9604-885c01367117 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:56.714279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:56.715729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:56.869181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 787/1562] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:56.870179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:56.873364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:56.893925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592366a9-8ad5-41b4-aaa3-aa0ad6eaece9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:56.916083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:56.916772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.917060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.917341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.918105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:56.918401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:56.921759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5707-d8ae-48a6-b359-f7d5db95d153 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:56.937272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:56.942866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:56.950303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.951259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.951838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.953169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:56.955109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 788/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:56.957181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fe85-fe82-488a-ad0f-860e0d554a22 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:56.958174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fe85-fe82-488a-ad0f-860e0d554a22 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:56.958630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fe85-fe82-488a-ad0f-860e0d554a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:56.962156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd926b-a631-410d-bc37-e2141d0d938b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.008005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a5e5fe85-fe82-488a-ad0f-860e0d554a22 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.010867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a5e5fe85-fe82-488a-ad0f-860e0d554a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.025393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 776/1564] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/users => generated 297 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.030708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 789/1565] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.030708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1c2cd-987f-4c5a-af49-6d6c0e88506f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.030708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1c2cd-987f-4c5a-af49-6d6c0e88506f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.030708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1c2cd-987f-4c5a-af49-6d6c0e88506f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.033116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.037410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.037885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.060523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5ae1c2cd-987f-4c5a-af49-6d6c0e88506f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.062640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5ae1c2cd-987f-4c5a-af49-6d6c0e88506f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.064807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.067639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.083129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 790/1566] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.083351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 777/1567] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.083571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0148d78-8a4b-480d-b7e6-44a6fab31141 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.083571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0148d78-8a4b-480d-b7e6-44a6fab31141 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.083571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0148d78-8a4b-480d-b7e6-44a6fab31141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.083830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.100442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7106e-4bc0-4fa4-8d26-b21e06a9c135 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.106307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e0148d78-8a4b-480d-b7e6-44a6fab31141 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.107179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.107706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e0148d78-8a4b-480d-b7e6-44a6fab31141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.107958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.108304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.108795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.109519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.111528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.117420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 791/1568] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.125865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.126155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.126433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.148786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.150422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.152828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 778/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.154550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.231669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 792/1570] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.232903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.253802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757641f1-5ce2-41fb-82c0-1993718dce85 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.257168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72220f2-6669-44ca-aa68-463748cb0d74 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.261297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.262259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.262726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.263091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.264200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:57.264603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.266957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1393a-7311-4235-ae01-60ae64fbe4c2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.267305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.267967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.268256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.268542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.269343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.272503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739eb72-50eb-41e3-a0f0-73a386785c7e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.283013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 793/1571] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:57.285275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 779/1572] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:57.286392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd492-116c-4994-b898-7b883a549530 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.286727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd492-116c-4994-b898-7b883a549530 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.286994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd492-116c-4994-b898-7b883a549530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.296905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.297578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.300103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.311434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-045fd492-116c-4994-b898-7b883a549530 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.312745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-045fd492-116c-4994-b898-7b883a549530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.327042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 780/1573] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.327042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911d76-6397-407a-a9f8-870adf2bdc1d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.327042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911d76-6397-407a-a9f8-870adf2bdc1d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.327042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae911d76-6397-407a-a9f8-870adf2bdc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.328267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.329477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.342249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 794/1574] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.343227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.353376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5fb8b-3307-4597-ad13-27ed8378017b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.358493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.359199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.359554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.359903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.360634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.362966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.363239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ae911d76-6397-407a-a9f8-870adf2bdc1d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.364913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ae911d76-6397-407a-a9f8-870adf2bdc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.375880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 781/1575] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.377395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.378454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.378750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.395779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 795/1576] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.399838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.409295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:57.410982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd7330-7f7e-46f0-a0c6-7d3bd8c110aa admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.411231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:57.417341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.418340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.418706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.418986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] PATH_INFO: `/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07/users/3d1ace728e9d41cf9338d2b8d6754098/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.422978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 782/1577] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:57.423904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.435164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875db28e-2dd7-41df-9ec3-0660329272f0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.437665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d5e7291c514b79bbbbcc95b5e13d07, user_id=3d1ace728e9d41cf9338d2b8d6754098, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.440386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.440531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.441503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.441865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.442137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.442774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.445219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.457546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 796/1578] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:57 2022] PUT /identity/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07/users/3d1ace728e9d41cf9338d2b8d6754098/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:57.458702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.483844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 783/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.484545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.540126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09d479-5307-47fd-acd2-7ce62926c195 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.556437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3071d-9bd1-40a3-9127-76710d5d81ad admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.561041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.561678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.561986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.562248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.562899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:57.563203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.564819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.566198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:57.568530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:57.588358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 784/1580] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:57.588358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.594959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434dc3-16c8-43d2-a628-5306af50eec3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.601239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.602034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.602414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.602854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.603586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:57.603981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.605778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.619651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 785/1581] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:57.619868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.620500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.620703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.620976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.621680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:57.622661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.624475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.627697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.638924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed213-2955-4eac-ba05-b7dff0033fc1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.639385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 797/1582] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:57.640721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.653451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.654222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.654674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.654785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b35aa-4b2c-4898-ae61-e955738f0c5b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.655262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.655831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.657793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.661806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.662538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.662891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.663406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.664163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:49:57.664618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.666713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.672072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 786/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:57.672835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.692217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 798/1584] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:49:57.692217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.692657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0294-7e0a-44cb-8532-fdb53b2ed9df admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.698454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.698690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c45cb-ab57-47bb-a1ce-3537e4a29773 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.699673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.699963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.700289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.701057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.703094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.703862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.704479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.704756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.705026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.705685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.707681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.723255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 787/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:57.723255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.737964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 799/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.737964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.748085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78b37b-6316-4fc2-8624-e2e9be546822 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.753368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.754357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.754708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.755049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.755838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.758384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.769317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffe841-b086-49ea-8288-eeaf9c2486c0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.777093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.778921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.779236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.779502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.780215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.782557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.804646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 788/1587] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.806011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.823808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 800/1588] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 393 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.823808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.903634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce338b-7dec-4091-91d4-ac7baa230b7c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.908928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa67cef-0ec5-45df-a33b-52c2a6e85aee admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.910135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.911013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.911540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.912094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.913249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.915762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.915985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:57.916720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:57.917177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:57.917441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:57.918854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:57.920885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:57.937080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 801/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:57.937968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:57.951529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b2f8-cd1f-4a45-8195-b46f8adf7e89 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:57.953072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 789/1590] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:57.954001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.030268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.030956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.031310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.031635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.032583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.034826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.035023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1fda8-8479-4202-849a-5e801417f2ca admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.040716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.041632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.042061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.042440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] PATH_INFO: `/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07/users/3d1ace728e9d41cf9338d2b8d6754098/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.042842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:58.045408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d5e7291c514b79bbbbcc95b5e13d07, user_id=3d1ace728e9d41cf9338d2b8d6754098, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.048986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.074386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 790/1591] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:57 2022] PUT /identity/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07/users/3d1ace728e9d41cf9338d2b8d6754098/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:58.074386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.081794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 802/1592] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:57 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:58.081794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.101207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:58.159729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce81e27-a197-47d0-be40-e00879e8920c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.164033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974ba3-32c2-45a7-8150-c4cbead56a5a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.167333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.168420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.169014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.169496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.169725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.170753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.171404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.171693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.172541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.173207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.174150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.175814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.189843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 791/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.191008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.205637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72970-6d35-401f-b333-3eaa4f4dcd49 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.209544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 803/1594] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:58.209544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.219226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.220387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.220788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.221108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.222023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.222253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76392481-7754-44a7-9685-06726dcaee9b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.224122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.229591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.230392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.230845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.231325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] PATH_INFO: `/v3/projects/2be7be1a9bb1487aac2c32871398119e/users/ccbb0b2070694235bf87a7b9a0c84dbe/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.252261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 792/1595] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/users => generated 297 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:58.253147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.264792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be7be1a9bb1487aac2c32871398119e, user_id=ccbb0b2070694235bf87a7b9a0c84dbe, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.266190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293000-1967-4708-943d-4808a40a8d1b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.268369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6eb3a-f138-4914-ae5a-fd8230331f55 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.271519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.272308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.272635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.272910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.273539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.275986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.291914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 804/1596] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:58 2022] PUT /identity/v3/projects/2be7be1a9bb1487aac2c32871398119e/users/ccbb0b2070694235bf87a7b9a0c84dbe/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:58.294094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.294517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.294851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.306569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 793/1597] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/users => generated 294 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:58.307494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.323240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40807db-eecc-4f4a-a248-4ec81f70617f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.332316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] MFA Rules not processed for user `3d1ace728e9d41cf9338d2b8d6754098`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:58.333982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:58.400365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.401454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.402356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.402908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.403721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.407340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.417696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 805/1598] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:58.419696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.431773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a35f2-7ef6-4bba-b07b-3f3b9d48f4db None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.437625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.438224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.438529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.438872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.439579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.441229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.455063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 794/1599] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/users => generated 303 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:49:58.456547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.465314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 806/1600] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.466802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.470850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51772b-e6cb-4385-864f-7aa198acd2e5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.477721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.478351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.478640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.478913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.479748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.481411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440da102-e4b5-406f-824a-1810fdffe7dd admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.481589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cee1d6-ce54-4bdc-a820-3eb4d6b56398 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.487973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.489132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.489609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.490069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.491703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.494477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.495219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 795/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.500222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.500505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.501284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.516916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 807/1602] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.519288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.526415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:58.527635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:58.535699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988edafb-70ae-4361-bc92-c00a123fd5b7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.541228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 796/1603] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:58.542271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.544058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.551358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.551798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.552153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.554065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.564878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.580093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 808/1604] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.580093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.586698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:58.592214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ee890-78f4-4e22-bad4-eb97f41cb749 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.603244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.604013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.604290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.604559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.605262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.606981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.615366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9146-bef3-45c4-8b83-8d9e2f016024 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.622731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.622731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.623028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 809/1605] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.623028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.624628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.624973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.638375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:58.642868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.643343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc88446-dfa0-4943-b97a-a78b043c5f3d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.644346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.650120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.650815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.651125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.651403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] PATH_INFO: `/v3/projects/eb4aab6e4d8f4582a2538ccd0004046c/users/e05967546ef245d2b88874f6cf3f84fd/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.664420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 797/1606] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:58.665289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.678314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc568-c52c-4299-82ef-df9a7323ea0a service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.681902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4aab6e4d8f4582a2538ccd0004046c, user_id=e05967546ef245d2b88874f6cf3f84fd, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.683960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.684649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.684992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.685329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.686086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.686235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.687932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.704598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 798/1607] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.705738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.714517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 810/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:58 2022] PUT /identity/v3/projects/eb4aab6e4d8f4582a2538ccd0004046c/users/e05967546ef245d2b88874f6cf3f84fd/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:58.714810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.720496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7a65-bc10-44d8-b87e-3c13820e466f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.797981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8395-b4d2-4d82-9d78-0450b615b82b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.801162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.801835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.802105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.802383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.812927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.813585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.814022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.814297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.814951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.816770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.830818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 799/1609] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.831878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.844606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd9cc-cfe8-496c-a043-7e0a587be772 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.852257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.853053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.854558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.854834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.856123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.858264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.876535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 800/1610] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.876535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.932889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e00b5-fcee-47f9-a0c1-83efb08717b7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.932889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.935172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.935540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.935540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.935540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.935540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.935989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.936080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d946ae7-a314-4584-8b73-e0e405128dea admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:58.936177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 811/1611] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:58.936177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:58.936337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 801/1612] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:49:58.936337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.936337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.936337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.941603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f67b-1636-4ed0-9078-08aa2766d023 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:58.949533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:58.950436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:58.950871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:58.951248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] PATH_INFO: `/v3/projects/df65a60541a8492a912cab1a8f243253/users/8dd2f6cc960242e2b01df2caf0aba2cd/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:58.989558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df65a60541a8492a912cab1a8f243253, user_id=8dd2f6cc960242e2b01df2caf0aba2cd, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:58.994837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:58.998082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:58.999658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.015116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 802/1613] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:59.016808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.026160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 812/1614] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:58 2022] PUT /identity/v3/projects/df65a60541a8492a912cab1a8f243253/users/8dd2f6cc960242e2b01df2caf0aba2cd/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:59.027397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.107081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:59.113093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68c1bc-1429-4932-8169-ae831719def2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.124472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.125396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.125783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.126243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] PATH_INFO: `/v3/projects/ef6b481491244111a479eb91e3594ae4/users/41bc35497fe94ababe0c824a429168d3/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.143382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95583d49-831e-48af-b4bd-0ccf61577c03 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.149239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6b481491244111a479eb91e3594ae4, user_id=41bc35497fe94ababe0c824a429168d3, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.153061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.178934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 813/1615] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:59 2022] PUT /identity/v3/projects/ef6b481491244111a479eb91e3594ae4/users/41bc35497fe94ababe0c824a429168d3/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:59.179778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.230800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.232077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.232359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.232632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.262547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:59.294479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88cf2-0aeb-482e-9dbd-8e39967b4763 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.306350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.307169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.307561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.307940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] PATH_INFO: `/v3/projects/a811b0090887457ea161b09786b2a640/users/202663b4d752486eb11b6e2bf9d2b984/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.315242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.316798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.335062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a811b0090887457ea161b09786b2a640, user_id=202663b4d752486eb11b6e2bf9d2b984, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.335202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 803/1616] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:59.335895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.338748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9a801-8a2f-4ebd-8889-493bf292b600 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.348219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e583d3a-1a96-4263-86ed-f212aeb9a523 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.353702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.354839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.354839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.354839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] PATH_INFO: `/v3/projects/73583cea62384146b3577e293c4a7703/users/1af1543fd46b449ba50211107ce3870c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.365065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 814/1617] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:59 2022] PUT /identity/v3/projects/a811b0090887457ea161b09786b2a640/users/202663b4d752486eb11b6e2bf9d2b984/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:59.367377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a087d-57a1-4e11-8b9d-d8b266650197 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.367817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a087d-57a1-4e11-8b9d-d8b266650197 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.368456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a087d-57a1-4e11-8b9d-d8b266650197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.372588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=73583cea62384146b3577e293c4a7703, user_id=1af1543fd46b449ba50211107ce3870c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.376223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.402361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-772a087d-57a1-4e11-8b9d-d8b266650197 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:59.403424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 804/1618] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:49:59 2022] PUT /identity/v3/projects/73583cea62384146b3577e293c4a7703/users/1af1543fd46b449ba50211107ce3870c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:49:59.403571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-772a087d-57a1-4e11-8b9d-d8b266650197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:59.404569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.535256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 815/1619] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:59.538561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020e9b-7827-4ab8-9d4a-b448a6500785 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.538855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020e9b-7827-4ab8-9d4a-b448a6500785 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.540298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020e9b-7827-4ab8-9d4a-b448a6500785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.543711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5da77-fde7-45af-8ebd-ee47a2c178cf admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.560410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.561482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.562087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.562534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.573836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c7020e9b-7827-4ab8-9d4a-b448a6500785 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:59.575412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c7020e9b-7827-4ab8-9d4a-b448a6500785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:59.582867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.585144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd8697-5050-4493-99a5-6a13a04fa277 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.591841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 816/1620] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:59.591841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.591841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.591841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.636651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 805/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:59.642651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.642942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.643200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.652054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] MFA Rules not processed for user `202663b4d752486eb11b6e2bf9d2b984`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:59.655652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:59.680841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:49:59.682332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:49:59.697358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 806/1622] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:59.698633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.719219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d46efb-692f-4e82-888f-c9011a4d1ef9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.727379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.728434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.729011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.729508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.741326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.743321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.758130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 807/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:59.759394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.772894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de36874-8ebc-4e5f-b3fe-f17a9356c16d service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.780311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.782883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.783174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.783449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.809723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:49:59.817023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.819613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.849738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 808/1624] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:59.850974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.875708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176402-4aa5-4125-b41e-8fae19faa906 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.883003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 817/1625] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:49:59.884496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:49:59.891042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.891673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.892161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.892315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.907515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.909597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:49:59.914051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfbb9d-3a73-4e46-848b-e66b54de5596 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:49:59.927502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:49:59.928393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:49:59.929108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:49:59.930032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:49:59.930696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:49:59.931836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 809/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:49:59.939079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.026111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 818/1627] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:49:59 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:00.030297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:00.045786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef307-8739-487c-b1d6-ba50d7730880 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:00.152996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:00.176019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:00.176942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.182513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.182801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.188006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:00.190310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.246471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 810/1628] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:00 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:00.251211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:00.287182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b21e1-6173-49ea-9f92-78633ef4762b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:00.297211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:00.297984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.298268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.298547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.299412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:00.301155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.342026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 819/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:00.346501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:00.364219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd758dc-f555-4e7b-8b30-8a03fe980d22 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:00.370030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:00.370934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.371289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.371637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] PATH_INFO: `/v3/projects/cb99fd71e5464f22a2e05b7e096b8188/users/2f78cc773df54c45aa270a59174f2277/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.397827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99fd71e5464f22a2e05b7e096b8188, user_id=2f78cc773df54c45aa270a59174f2277, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:00.401118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.425586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 811/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:00 2022] PUT /identity/v3/projects/cb99fd71e5464f22a2e05b7e096b8188/users/2f78cc773df54c45aa270a59174f2277/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:00.429927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:00.445426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b256-ad06-48e7-b892-d7281e8a6304 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:00.603588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:00.604162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.604443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.604719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.605395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:00.607077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14d3b-398e-4305-a47d-f5c2cb965867 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.622541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 820/1631] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:00.626764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:00.655753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56590d-4b00-4cb7-9025-9ac31f5fd00d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:00.679952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:00.683559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.684906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.685296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] PATH_INFO: `/v3/projects/cb99fd71e5464f22a2e05b7e096b8188/users/2f78cc773df54c45aa270a59174f2277/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.697026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99fd71e5464f22a2e05b7e096b8188, user_id=2f78cc773df54c45aa270a59174f2277, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:00.699945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:00.735823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 812/1632] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:00 2022] PUT /identity/v3/projects/cb99fd71e5464f22a2e05b7e096b8188/users/2f78cc773df54c45aa270a59174f2277/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:00.747539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:00.748117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:00.748401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:00.786110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] MFA Rules not processed for user `2f78cc773df54c45aa270a59174f2277`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:00.787287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:00.973795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 821/1633] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:00.987570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:01.132415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5032-396c-4444-bebf-9155d30fac55 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:01.149630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:01.151285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:01.151680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:01.153010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:01.176368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:01.182719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57213f0-b0f2-4bb6-9684-715c8e2ed634 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:01.205099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 813/1634] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:01.264003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:01.339112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:01.414317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f2b94-864b-4b79-8ca3-f39c7370a799 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:01.424117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:01.426174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:01.426843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:01.427948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] PATH_INFO: `/v3/users/202663b4d752486eb11b6e2bf9d2b984` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:01.431615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] RBAC: Authorizing `identity:delete_user(user_id=202663b4d752486eb11b6e2bf9d2b984)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:01.434705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: WARNING py.warnings [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:50:01.434705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: warnings.warn(msg) Sep 06 11:50:01.434705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: Sep 06 11:50:01.438659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:01.588711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202663b4d752486eb11b6e2bf9d2b984'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:01.601469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202663b4d752486eb11b6e2bf9d2b984'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:01.637116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202663b4d752486eb11b6e2bf9d2b984'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:01.639380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202663b4d752486eb11b6e2bf9d2b984'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:01.657446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202663b4d752486eb11b6e2bf9d2b984'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:01.666208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 822/1635] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:01 2022] DELETE /identity/v3/users/202663b4d752486eb11b6e2bf9d2b984 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:01.670526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf3d5-e68c-41d9-a664-828ac08e5cfb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:01.671356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf3d5-e68c-41d9-a664-828ac08e5cfb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:01.671742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaf3d5-e68c-41d9-a664-828ac08e5cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:01.698286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-daeaf3d5-e68c-41d9-a664-828ac08e5cfb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:01.703299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-daeaf3d5-e68c-41d9-a664-828ac08e5cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:01.802243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 814/1636] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:01.843400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:01.845131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:02.075341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71d03-cf52-4ecb-9ae5-a597ef66126c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:02.088624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:02.089332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:02.089617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:02.090191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:02.126441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:02.133911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:02.173582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 823/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:02.860075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:02.861067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:02.862162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:02.942665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] MFA Rules not processed for user `1af1543fd46b449ba50211107ce3870c`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:02.945333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:03.034078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:03.109276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 815/1638] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:03.114554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:03.117378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:03.140876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353048-0e20-43f6-878a-183b35acd171 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:03.170326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:03.171039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:03.171327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:03.171603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:03.172265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:03.174297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:03.245883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:03.258662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 824/1639] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:03 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:03.264897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d6988-d8e4-4ed2-9368-41de15e2e08a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:03.528711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:03.529440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:03.529638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:03.529939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:03.530776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:03.532966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:03.596936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:03.612673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 816/1640] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:03 2022] POST /identity/v3/users => generated 309 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:03.666443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3b9eb-810b-4a4e-94cc-12882ad23b2e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:03.676700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:03.677839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:03.678393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:03.679011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:03.681435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:03.684304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:03.731655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:03.735296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 825/1641] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:03.749197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aca936-fa49-4fc9-b12f-1da39ca3a31f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:03.772736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:03.774373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:03.774770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:03.775063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] PATH_INFO: `/v3/projects/33a07c03900049c59e6f2952f01e7d6e/users/c129dd1786a140cdbeeec62251ed0989/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:03.806243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a07c03900049c59e6f2952f01e7d6e, user_id=c129dd1786a140cdbeeec62251ed0989, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:03.812287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:03.843489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 817/1642] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:03 2022] PUT /identity/v3/projects/33a07c03900049c59e6f2952f01e7d6e/users/c129dd1786a140cdbeeec62251ed0989/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:03.847760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:03.867743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c30b-e92f-4c99-8377-948cd6a25c2c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:03.901239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:04.008057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:04.008960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:04.009346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:04.009738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:04.010937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:04.013765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:04.042628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:04.045473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 826/1643] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:50:04.067311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36ae53-7ad0-43cd-a68f-73275e64eac9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:04.079952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:04.080973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:04.081420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:04.083102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] PATH_INFO: `/v3/projects/33a07c03900049c59e6f2952f01e7d6e/users/c129dd1786a140cdbeeec62251ed0989/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:04.087168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a07c03900049c59e6f2952f01e7d6e, user_id=c129dd1786a140cdbeeec62251ed0989, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:04.091335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:04.132031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 818/1644] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:04 2022] PUT /identity/v3/projects/33a07c03900049c59e6f2952f01e7d6e/users/c129dd1786a140cdbeeec62251ed0989/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:04.749777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:04.753100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:04.781735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d50bd8-9430-4402-b4c3-c23ff30aadd0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:04.922064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:04.922984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:04.923423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:04.923797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] PATH_INFO: `/v3/projects/a811b0090887457ea161b09786b2a640` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:04.926011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a811b0090887457ea161b09786b2a640)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:04.938703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:05.071869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a811b0090887457ea161b09786b2a640', 'request_id': 'req-9c20505c-329f-40ed-a69c-b71cb23712bd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.092322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a811b0090887457ea161b09786b2a640', 'request_id': 'req-9c20505c-329f-40ed-a69c-b71cb23712bd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.104876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a811b0090887457ea161b09786b2a640', 'request_id': 'req-9c20505c-329f-40ed-a69c-b71cb23712bd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.144220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] The token cache is being invalidate because project a811b0090887457ea161b09786b2a640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:05.144524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.147801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.154287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 827/1645] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:04 2022] DELETE /identity/v3/projects/a811b0090887457ea161b09786b2a640 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:05.158445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:05.160372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:05.192611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612c8cc-0af2-422c-b965-e90677cd8b7f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:05.320459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:05.321328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:05.321766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:05.322138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] PATH_INFO: `/v3/users/2f78cc773df54c45aa270a59174f2277` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:05.324414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f78cc773df54c45aa270a59174f2277)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:05.328295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:05.409529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:05.434261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f78cc773df54c45aa270a59174f2277'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.434983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f78cc773df54c45aa270a59174f2277'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.466675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f78cc773df54c45aa270a59174f2277'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.480034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f78cc773df54c45aa270a59174f2277'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.480668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41da5d41-cd66-48c4-b20a-b7642b0d6e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f78cc773df54c45aa270a59174f2277'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:05.508595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 819/1646] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:05 2022] DELETE /identity/v3/users/2f78cc773df54c45aa270a59174f2277 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:07.141165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:07.164979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20505c-329f-40ed-a69c-b71cb23712bd admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:07.290642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae97-9567-480e-aab1-7580b1cccea1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.291547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae97-9567-480e-aab1-7580b1cccea1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.292052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae97-9567-480e-aab1-7580b1cccea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.311125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:07.348709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0db0ae97-9567-480e-aab1-7580b1cccea1 None None] MFA Rules not processed for user `e235718ad1da42ebb209fb4b13626ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:07.352100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0db0ae97-9567-480e-aab1-7580b1cccea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:07.369176 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:07.370909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.372875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.373299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] PATH_INFO: `/v3/projects/cb99fd71e5464f22a2e05b7e096b8188` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.375714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb99fd71e5464f22a2e05b7e096b8188)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:07.382736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:07.448114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 820/1647] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:07.461889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9599d-b3e8-4da2-8389-2ccf33e14dfb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.465917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9599d-b3e8-4da2-8389-2ccf33e14dfb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.466573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9599d-b3e8-4da2-8389-2ccf33e14dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.546379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d9e9599d-b3e8-4da2-8389-2ccf33e14dfb None None] MFA Rules not processed for user `c129dd1786a140cdbeeec62251ed0989`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:07.554390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d9e9599d-b3e8-4da2-8389-2ccf33e14dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:07.589955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb99fd71e5464f22a2e05b7e096b8188', 'request_id': 'req-d4eb265f-6dd2-4c81-9b26-9fba8a863366'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:07.602190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb99fd71e5464f22a2e05b7e096b8188', 'request_id': 'req-d4eb265f-6dd2-4c81-9b26-9fba8a863366'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:07.631294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb99fd71e5464f22a2e05b7e096b8188', 'request_id': 'req-d4eb265f-6dd2-4c81-9b26-9fba8a863366'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:07.666681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] The token cache is being invalidate because project cb99fd71e5464f22a2e05b7e096b8188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:07.669135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:07.670883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d4eb265f-6dd2-4c81-9b26-9fba8a863366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:07.679490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 828/1648] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:07 2022] DELETE /identity/v3/projects/cb99fd71e5464f22a2e05b7e096b8188 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:07.684640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f601-f649-4a6d-804a-e460035ddb74 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.684928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f601-f649-4a6d-804a-e460035ddb74 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.685203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f601-f649-4a6d-804a-e460035ddb74 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.691662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 821/1649] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:07.693227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126005b-4faa-4481-a983-7957d6d2b342 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.696170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126005b-4faa-4481-a983-7957d6d2b342 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.696603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126005b-4faa-4481-a983-7957d6d2b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.697837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 829/1650] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 6 11:50:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:07.699803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.700098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:07.700366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:07.754982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1126005b-4faa-4481-a983-7957d6d2b342 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:07.758250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1126005b-4faa-4481-a983-7957d6d2b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:07.789239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:07.792718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:07.836024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:07.981427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 822/1651] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:07.998642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:07.998940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.000152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.043360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:08.046896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:08.062582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 830/1652] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:08.078081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.088847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 823/1653] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:08.090915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.107559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4106c-ddca-445f-8e12-b2a9774843f6 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.111139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03524-b702-43c4-ac26-fcea336e7f68 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.120649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.122411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.122742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.123000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.123266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.123643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.123905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.123998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:08.124172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.124323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.126185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.131959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:08.169577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 824/1654] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:08.171515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.199114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533bad9-3320-43c6-bb16-dee1ee8aa8b2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.214145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.214790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.215066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.215333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.219568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.227460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.291058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:08.297727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.300916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.338828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 825/1655] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:08 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:08.339947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.342886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 831/1656] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:08.548626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cc8c3-4f67-4255-b612-0a7168e7dd96 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.565761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.566494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.566777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.567047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.567911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.571135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.644950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 826/1657] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:08 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:08.649559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.668985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce54f-b57e-4ade-8816-cb0a069fb0d9 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.692554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.693201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.694241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.694419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.694990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.696631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.740541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 832/1658] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:08.741829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.767000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200f26-363e-470e-a61d-5e478c1c2653 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.786473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.787115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.787924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.788197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.788867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.790548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd23028-81f1-45a2-932b-4b9e00bc2b6f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:08.837148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 827/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:08.838766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:08.849369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629264db-f4bd-4fb1-b6b3-c80f328239b0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:08.886281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:08.886981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:08.887267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:08.887545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] PATH_INFO: `/v3/projects/9a95856046444c1eba3668d4600c9715/users/0ccc47ec67754cff9009686f12abe2f6/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:08.944164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a95856046444c1eba3668d4600c9715, user_id=0ccc47ec67754cff9009686f12abe2f6, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:08.951013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:09.000574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 833/1660] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:08 2022] PUT /identity/v3/projects/9a95856046444c1eba3668d4600c9715/users/0ccc47ec67754cff9009686f12abe2f6/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:09.013045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:09.013541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:09.013897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:09.117710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] MFA Rules not processed for user `0ccc47ec67754cff9009686f12abe2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:09.117710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:09.299496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:09.364578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 828/1661] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:09.366753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:09.477573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c907a-a89d-448a-a79e-0323db0a46fd admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:09.485357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:09.486284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:09.486881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:09.487379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:09.488204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:09.492576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:09.551740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 834/1662] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:09 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:09.556806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:09.574362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5371-9adc-417a-9c9e-1a03cec55c91 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:09.757261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:09.758221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:09.759046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:09.760363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:09.765319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:09.769456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:09.820949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 829/1663] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:09 2022] POST /identity/v3/users => generated 298 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:09.824963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:09.852189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a1cf-32a8-4b7e-a9db-612db2ba7d30 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:09.878020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:09.879139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:09.879605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:09.880014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:09.880870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:09.896503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:09.925700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 835/1664] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:09.926969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:09.945701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfea6b-435b-4b2e-a797-628679962412 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:09.966770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:09.967781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:09.968686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:09.969216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] PATH_INFO: `/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651/users/746a0118414c4940b214d687e9322fc1/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.007193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ea4f68232b4d33ab4b593e3bd88651, user_id=746a0118414c4940b214d687e9322fc1, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:10.010393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:10.049923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 830/1665] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:09 2022] PUT /identity/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651/users/746a0118414c4940b214d687e9322fc1/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:10.054216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:10.069852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5434-0a56-4621-a431-3c1bb2e4e7d8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:10.221773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:10.222562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.222663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.223045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.223711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:10.225927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cd990-cb25-45fa-9ed4-c038efec69e0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:10.250706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 836/1666] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:10.252892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:10.279616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7081d-2525-492d-9f9e-c25f3c28efc5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:10.310829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:10.311783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.312257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.312765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] PATH_INFO: `/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651/users/746a0118414c4940b214d687e9322fc1/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.315115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ea4f68232b4d33ab4b593e3bd88651, user_id=746a0118414c4940b214d687e9322fc1, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:10.319298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:10.340388 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 831/1667] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:10 2022] PUT /identity/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651/users/746a0118414c4940b214d687e9322fc1/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:10.351391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc5409-e5c9-4b4e-b60e-d45ef1d6800f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.351712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc5409-e5c9-4b4e-b60e-d45ef1d6800f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.351964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc5409-e5c9-4b4e-b60e-d45ef1d6800f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.390322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1acc5409-e5c9-4b4e-b60e-d45ef1d6800f None None] MFA Rules not processed for user `746a0118414c4940b214d687e9322fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:10.392573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1acc5409-e5c9-4b4e-b60e-d45ef1d6800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:10.424753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:10.558039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 837/1668] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:10.573438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:10.575379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:10.693453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea681-a016-4ac0-884c-413c8145e336 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:10.697970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:10.699523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.699862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.700314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.713441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:10.715450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd352fc-d028-4ae9-b257-06e105c35a65 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:10.735319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 832/1669] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:10.790727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bc5e-3242-4f6b-bd61-9e8f9d9dfd69 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.791010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bc5e-3242-4f6b-bd61-9e8f9d9dfd69 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.791291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bc5e-3242-4f6b-bd61-9e8f9d9dfd69 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.792977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 838/1670] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 11:50:10.891253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedba4d-9a4c-4425-8ab6-97090ea6b559 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:10.891518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedba4d-9a4c-4425-8ab6-97090ea6b559 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:10.891830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedba4d-9a4c-4425-8ab6-97090ea6b559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:10.933763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2bedba4d-9a4c-4425-8ab6-97090ea6b559 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:10.934986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2bedba4d-9a4c-4425-8ab6-97090ea6b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:10.954956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9bc5e-3242-4f6b-bd61-9e8f9d9dfd69 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:11.054096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9bc5e-3242-4f6b-bd61-9e8f9d9dfd69 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:11.060362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:11.061191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:11.062357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:11.062765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] PATH_INFO: `/v3/users/0ccc47ec67754cff9009686f12abe2f6` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:11.064552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ccc47ec67754cff9009686f12abe2f6)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:11.066997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:11.072055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:11.124104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 833/1671] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:50:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:11.179305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ccc47ec67754cff9009686f12abe2f6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:11.189294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ccc47ec67754cff9009686f12abe2f6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:11.223218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ccc47ec67754cff9009686f12abe2f6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:11.224126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ccc47ec67754cff9009686f12abe2f6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:11.234431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ccc47ec67754cff9009686f12abe2f6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:11.248214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 839/1672] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:10 2022] DELETE /identity/v3/users/0ccc47ec67754cff9009686f12abe2f6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:11.251305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:11.251587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:11.251861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:11.270423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:11.273161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:11.311281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:11.312722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:11.388479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:11.504661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433820f7-bf29-4782-aa0f-96b746c3d4e7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:11.530780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:11.531615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:11.533446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 834/1673] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:11.541333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:11.541333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:11.546311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:11.583111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:11.586315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:11.612617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 840/1674] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:11.675337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:11.678263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:11.856196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abbe48-6ab0-45f4-8648-8eaff61e6d8c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:11.874172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:11.875556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:11.876049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:11.876316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:11.890001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:11.894410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:11.915653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 835/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:14.079082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:14.080759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:14.118937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03161cc4-8101-44a0-8b59-e7ca1ccd4e3b service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:14.128581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:14.130306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:14.136948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:14.139400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:14.140099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:14.142717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] PATH_INFO: `/v3/users/1af1543fd46b449ba50211107ce3870c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:14.146124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af1543fd46b449ba50211107ce3870c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:14.153797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:14.169059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62862256-947f-4947-9530-32b6dd05303c service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:14.187644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:14.188549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:14.190030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:14.190482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] PATH_INFO: `/v3/projects/9a95856046444c1eba3668d4600c9715` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:14.194070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a95856046444c1eba3668d4600c9715)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:14.195926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: WARNING py.warnings [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 11:50:14.195926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: warnings.warn(msg) Sep 06 11:50:14.195926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: Sep 06 11:50:14.201976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:14.280608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af1543fd46b449ba50211107ce3870c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.293664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af1543fd46b449ba50211107ce3870c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.319962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af1543fd46b449ba50211107ce3870c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.321463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af1543fd46b449ba50211107ce3870c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.333852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af1543fd46b449ba50211107ce3870c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.343220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 841/1676] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:14 2022] DELETE /identity/v3/users/1af1543fd46b449ba50211107ce3870c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:14.377360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a95856046444c1eba3668d4600c9715', 'request_id': 'req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.399989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a95856046444c1eba3668d4600c9715', 'request_id': 'req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.416977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a95856046444c1eba3668d4600c9715', 'request_id': 'req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.461691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] The token cache is being invalidate because project 9a95856046444c1eba3668d4600c9715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:14.462008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.462689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.466740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 836/1677] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:14 2022] DELETE /identity/v3/projects/9a95856046444c1eba3668d4600c9715 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:14.471945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:14.474496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:14.695598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803de22-4ba9-42bc-a223-4480e13048e3 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:14.702900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:14.703970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:14.704458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:14.704937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] PATH_INFO: `/v3/users/746a0118414c4940b214d687e9322fc1` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:14.707455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=746a0118414c4940b214d687e9322fc1)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:14.714270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:14.908816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:14.946461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746a0118414c4940b214d687e9322fc1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:14.967873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746a0118414c4940b214d687e9322fc1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:15.011849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746a0118414c4940b214d687e9322fc1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:15.014830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746a0118414c4940b214d687e9322fc1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:15.062228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746a0118414c4940b214d687e9322fc1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:15.101714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 842/1678] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:14 2022] DELETE /identity/v3/users/746a0118414c4940b214d687e9322fc1 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:16.587276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:16.588837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:16.611116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:16.696468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e09d2a-9a5a-4cb2-8f73-7cfb1dad1405 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:16.822595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:16.824011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:16.824298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:16.825455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] PATH_INFO: `/v3/projects/73583cea62384146b3577e293c4a7703` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:16.828443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] RBAC: Authorizing `identity:delete_project(project_id=73583cea62384146b3577e293c4a7703)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:16.831253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:16.962321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73583cea62384146b3577e293c4a7703', 'request_id': 'req-1c4a0742-8074-4535-80c2-69a01c4703db'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:16.976922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73583cea62384146b3577e293c4a7703', 'request_id': 'req-1c4a0742-8074-4535-80c2-69a01c4703db'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:16.986109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73583cea62384146b3577e293c4a7703', 'request_id': 'req-1c4a0742-8074-4535-80c2-69a01c4703db'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.026733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] The token cache is being invalidate because project 73583cea62384146b3577e293c4a7703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:17.027041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.029454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.034770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 837/1679] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:16 2022] DELETE /identity/v3/projects/73583cea62384146b3577e293c4a7703 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:17.039368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:17.041291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:17.063428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b14e-f95d-4ea0-a179-e47dee31ee8d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:17.229859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:17.233029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.237036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.241745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] PATH_INFO: `/v3/users/c129dd1786a140cdbeeec62251ed0989` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.255852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c129dd1786a140cdbeeec62251ed0989)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:17.259312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:17.327907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:17.329495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:17.370985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0742-8074-4535-80c2-69a01c4703db admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:17.389331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:17.390866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.391151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.391420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] PATH_INFO: `/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.407431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ea4f68232b4d33ab4b593e3bd88651)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:17.419066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:17.444369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c129dd1786a140cdbeeec62251ed0989'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.463461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c129dd1786a140cdbeeec62251ed0989'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.488510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c129dd1786a140cdbeeec62251ed0989'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.489275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c129dd1786a140cdbeeec62251ed0989'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.510115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c129dd1786a140cdbeeec62251ed0989'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.516277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:17.528130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ea4f68232b4d33ab4b593e3bd88651', 'request_id': 'req-07009e0b-1e79-4116-b0a6-44ac56deec85'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.534479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 843/1680] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:17 2022] DELETE /identity/v3/users/c129dd1786a140cdbeeec62251ed0989 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:17.534479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:17.534479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:17.543538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ea4f68232b4d33ab4b593e3bd88651', 'request_id': 'req-07009e0b-1e79-4116-b0a6-44ac56deec85'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.575997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ea4f68232b4d33ab4b593e3bd88651', 'request_id': 'req-07009e0b-1e79-4116-b0a6-44ac56deec85'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.615666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] The token cache is being invalidate because project f5ea4f68232b4d33ab4b593e3bd88651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:17.617055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.617996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-07009e0b-1e79-4116-b0a6-44ac56deec85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.623244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 838/1681] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:17 2022] DELETE /identity/v3/projects/f5ea4f68232b4d33ab4b593e3bd88651 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:17.624741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271245c-d852-41ca-aed8-ead69e24af39 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.625162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271245c-d852-41ca-aed8-ead69e24af39 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.625569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271245c-d852-41ca-aed8-ead69e24af39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.642189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:17.682576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e2136-73f0-46ad-a777-d822152ff496 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:17.688659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:17.689316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.689593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.689939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] PATH_INFO: `/v3/users/739ec5079888451d93c4bcd38812d54b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.691133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6271245c-d852-41ca-aed8-ead69e24af39 None None] MFA Rules not processed for user `ccbb0b2070694235bf87a7b9a0c84dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:17.691323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=739ec5079888451d93c4bcd38812d54b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:17.692374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6271245c-d852-41ca-aed8-ead69e24af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:17.694208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:17.775549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '739ec5079888451d93c4bcd38812d54b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.789512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '739ec5079888451d93c4bcd38812d54b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.814056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '739ec5079888451d93c4bcd38812d54b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.816552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '739ec5079888451d93c4bcd38812d54b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.836601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91986ab0-6ea4-4969-94cc-24aed74cd446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '739ec5079888451d93c4bcd38812d54b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:17.853545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 844/1682] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:17 2022] DELETE /identity/v3/users/739ec5079888451d93c4bcd38812d54b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:17.854911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97540a-487d-4ace-b3ef-ef4d79f3cbe9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.855206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97540a-487d-4ace-b3ef-ef4d79f3cbe9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.855476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97540a-487d-4ace-b3ef-ef4d79f3cbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.862494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 839/1683] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:17.864900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ee57-62bf-44fe-a1b6-8a14bbd5d2a5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:17.865248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ee57-62bf-44fe-a1b6-8a14bbd5d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:17.865519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ee57-62bf-44fe-a1b6-8a14bbd5d2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:17.886333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-dd97540a-487d-4ace-b3ef-ef4d79f3cbe9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:17.890109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-dd97540a-487d-4ace-b3ef-ef4d79f3cbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:17.934594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b716ee57-62bf-44fe-a1b6-8a14bbd5d2a5 None None] MFA Rules not processed for user `8dd2f6cc960242e2b01df2caf0aba2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:17.939099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b716ee57-62bf-44fe-a1b6-8a14bbd5d2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.047118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 845/1684] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.055811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf6a31-463b-4d94-a438-9c5856ffb599 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.055811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf6a31-463b-4d94-a438-9c5856ffb599 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.055811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf6a31-463b-4d94-a438-9c5856ffb599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.087914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-f8bf6a31-463b-4d94-a438-9c5856ffb599 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.089382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f8bf6a31-463b-4d94-a438-9c5856ffb599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.112795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 840/1685] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.114345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528253d2-9c0c-4e99-8712-4b22293196a4 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.115662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528253d2-9c0c-4e99-8712-4b22293196a4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.116723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528253d2-9c0c-4e99-8712-4b22293196a4 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.118749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 841/1686] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 6 11:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:18.120067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926981e-4fc1-4228-8f08-66afc3ef8eae None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.120497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926981e-4fc1-4228-8f08-66afc3ef8eae None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.120938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926981e-4fc1-4228-8f08-66afc3ef8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.129141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 846/1687] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.129141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9100-d5fc-4335-9d54-9c8297b45594 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.129141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9100-d5fc-4335-9d54-9c8297b45594 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.129141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9100-d5fc-4335-9d54-9c8297b45594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.170970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6926981e-4fc1-4228-8f08-66afc3ef8eae None None] MFA Rules not processed for user `e235718ad1da42ebb209fb4b13626ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.172617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6926981e-4fc1-4228-8f08-66afc3ef8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.173588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4bee9100-d5fc-4335-9d54-9c8297b45594 None None] MFA Rules not processed for user `e05967546ef245d2b88874f6cf3f84fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.175704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4bee9100-d5fc-4335-9d54-9c8297b45594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.237783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:18.275637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 847/1688] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.277069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15267-4d0a-4dd0-98db-99978d868e27 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.277463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15267-4d0a-4dd0-98db-99978d868e27 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.277882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15267-4d0a-4dd0-98db-99978d868e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.322275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:18.324844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-aba15267-4d0a-4dd0-98db-99978d868e27 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.326559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aba15267-4d0a-4dd0-98db-99978d868e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.343377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 842/1689] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.344972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.345266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.345546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.352228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 848/1690] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.355498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.355800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.356065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.397047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.398223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.404350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] MFA Rules not processed for user `8dd2f6cc960242e2b01df2caf0aba2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:18.405559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.429291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 843/1691] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.429291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:18.432236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.562997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 849/1692] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 6 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:18.564227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:18.565937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.573989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a024d-f79d-4971-9c7e-fbd2813dac60 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:18.586236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:18.586944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.587223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.587493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.594929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f3b7-445d-4734-8a1b-666beaddee4a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:18.602365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:18.603132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.603680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.606822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] PATH_INFO: `/v3/projects/33a07c03900049c59e6f2952f01e7d6e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.606966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:18.608407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a07c03900049c59e6f2952f01e7d6e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:18.608520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:18.610560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:18.633181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 844/1693] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:18.636340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:18.639041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.687591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bd6c-232d-42b7-9ad6-7c6e6f001a2e service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:18.708700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:18.709499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.709795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.710063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.730112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:18.731776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:18.759224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 845/1694] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:18.764259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:18.774124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a07c03900049c59e6f2952f01e7d6e', 'request_id': 'req-67b1b405-7094-491a-a894-4669e63c238b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:18.828729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67f09-4237-485a-97e2-30d80f06edf1 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:18.829332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a07c03900049c59e6f2952f01e7d6e', 'request_id': 'req-67b1b405-7094-491a-a894-4669e63c238b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:18.860626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a07c03900049c59e6f2952f01e7d6e', 'request_id': 'req-67b1b405-7094-491a-a894-4669e63c238b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:18.942884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] The token cache is being invalidate because project 33a07c03900049c59e6f2952f01e7d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:18.946793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:18.950195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:18.960189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 850/1695] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:18 2022] DELETE /identity/v3/projects/33a07c03900049c59e6f2952f01e7d6e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:18.966552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:18.973489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:18.982511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:18.983356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:18.983462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:18.983691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:18.984354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:18.984657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:18.986399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea6946-0c8b-4cc5-8b22-5d636ad3d956 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.022805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 846/1696] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:19.031139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.032150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.032150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.087673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:19.088902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.113197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 847/1697] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:19.115579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.118701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.148134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08d774-e021-4c4f-bedf-a49b06f48734 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.160837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.161472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.161771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.162098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.169211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.171309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014346-39d1-492e-ba7b-8aca481b5295 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.231597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:19.277349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 848/1698] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:19.279543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563104-cc84-46b3-99b3-5195d6b064bb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.283622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563104-cc84-46b3-99b3-5195d6b064bb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.285233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563104-cc84-46b3-99b3-5195d6b064bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.338548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b405-7094-491a-a894-4669e63c238b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.371299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-75563104-cc84-46b3-99b3-5195d6b064bb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:19.372545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-75563104-cc84-46b3-99b3-5195d6b064bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.539986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:19.575995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.582877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.582877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.583456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.588282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.618235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 849/1699] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:19.620155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75550-a794-41ed-becc-ed0f4c352ab6 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.620452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75550-a794-41ed-becc-ed0f4c352ab6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.620737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75550-a794-41ed-becc-ed0f4c352ab6 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.629449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 850/1700] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 6 11:50:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:19.630398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a75550-a794-41ed-becc-ed0f4c352ab6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.669480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a75550-a794-41ed-becc-ed0f4c352ab6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.675516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.676521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.676939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.677242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.679168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.684014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.739284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 851/1701] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:19.742950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.791725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8d14-bc83-4aa2-904b-f938848d49e5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.809276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.810551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.810979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.811700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.812571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:19.813386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.820932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.822580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.822702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e78912-265a-426f-9990-3aa4e13232f2 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.843665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 851/1702] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 883 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:19.846041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.849888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.850491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 852/1703] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:19.862199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.894647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3a072-2161-49f0-bf5c-8b9f4e62cad6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.901217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:19.902707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.913833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.914603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.914983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.915356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.916176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.918884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:19.926225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 852/1704] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:19.927447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.961247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 853/1705] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:19.964843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:19.976856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:19.977299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ae65d-94b7-4e68-b972-e58f172ebf5f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:19.988414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:19.989034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:19.989306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:19.989572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:19.990959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:19.994252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.079699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 853/1706] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:19 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:20.085004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:20.268589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:20.270661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fe0c0-bae1-42f2-8232-2e9bed2947ba admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:20.298295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d89f3e-a24a-4b32-a2a5-5a241726edb6 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:20.314966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:20.315683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.315965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.316332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.317199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:20.320995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.346876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 854/1707] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:20.347810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:20.354925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:20.394143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100df9d2-3472-4469-a362-08ab7d08df12 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:20.407633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:20.408275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.408565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.408827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] PATH_INFO: `/v3/projects/83ee0f5bbd6a485991984fe0bf4ffd44` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.410597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ee0f5bbd6a485991984fe0bf4ffd44)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:20.412706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.475721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:20.476579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.476867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.477143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.532373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ee0f5bbd6a485991984fe0bf4ffd44', 'request_id': 'req-930b5f6b-d8be-4059-b850-0cba79f74cd8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:20.547248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ee0f5bbd6a485991984fe0bf4ffd44', 'request_id': 'req-930b5f6b-d8be-4059-b850-0cba79f74cd8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:20.566990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ee0f5bbd6a485991984fe0bf4ffd44', 'request_id': 'req-930b5f6b-d8be-4059-b850-0cba79f74cd8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:20.600675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] The token cache is being invalidate because project 83ee0f5bbd6a485991984fe0bf4ffd44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:20.601372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:20.603898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:20.611096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 855/1708] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:20 2022] DELETE /identity/v3/projects/83ee0f5bbd6a485991984fe0bf4ffd44 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:20.618870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:20.619657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:20.623518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:20.651276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:20.653104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.669066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 854/1709] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:20.670456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:20.697979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d479f2-cc30-4888-b0a2-c16c70f223f6 service glance] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:20.825589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b5f6b-d8be-4059-b850-0cba79f74cd8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:20.839331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:20.839977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.840265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.840627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.841492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:20.848061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286ffa-0dd5-4eba-8bd3-1018e9393a86 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.880341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:20.882498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.884194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.884603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] PATH_INFO: `/v3/projects/eb53ec5b53bf4cd385ea1088a392e2d5/users/5e6049ea6c874b8e8623a9229ab46c3c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.893295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 856/1710] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:20 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:20.894748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:20.895040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:20.895317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:20.925044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb53ec5b53bf4cd385ea1088a392e2d5, user_id=5e6049ea6c874b8e8623a9229ab46c3c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:20.931964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:20.959714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] MFA Rules not processed for user `41bc35497fe94ababe0c824a429168d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:20.961358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:20.964043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 855/1711] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:20 2022] PUT /identity/v3/projects/eb53ec5b53bf4cd385ea1088a392e2d5/users/5e6049ea6c874b8e8623a9229ab46c3c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:20.974024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:20.976768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.072884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5716a-6932-4fa9-8c7a-168eccd3347a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:21.086924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:21.088802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.089090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.090251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] PATH_INFO: `/v3/users/3d1ace728e9d41cf9338d2b8d6754098` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.094404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:21.097326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1ace728e9d41cf9338d2b8d6754098)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:21.104837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:21.142981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 857/1712] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:21.142981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:21.144643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.215721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1ace728e9d41cf9338d2b8d6754098'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:21.216392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1ace728e9d41cf9338d2b8d6754098'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:21.239846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:21.249499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1ace728e9d41cf9338d2b8d6754098'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:21.250923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9087131-4687-4d91-8889-26eeb7c2ebce None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:21.260816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1ace728e9d41cf9338d2b8d6754098'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:21.260816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1ace728e9d41cf9338d2b8d6754098'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:21.263684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:21.264544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.265122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.265755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.269034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.285185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 856/1713] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:20 2022] DELETE /identity/v3/users/3d1ace728e9d41cf9338d2b8d6754098 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:21.286126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:21.291660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.429042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:21.435360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2619f-cbb2-4311-bd68-0c12a2238fb0 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:21.460670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 858/1714] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:21.462163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155bafa-e6da-4f52-bf00-65025650cc6e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.462457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155bafa-e6da-4f52-bf00-65025650cc6e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.462734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155bafa-e6da-4f52-bf00-65025650cc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.489277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b12eb-d5fd-42c9-a3a1-c93e9da4423d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:21.506770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:21.507525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.510480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.510876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.512061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:21.518023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d155bafa-e6da-4f52-bf00-65025650cc6e None None] MFA Rules not processed for user `5e6049ea6c874b8e8623a9229ab46c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:21.519141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff86324-2b9e-4118-b698-e62f6a6f7e16 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:21.519372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d155bafa-e6da-4f52-bf00-65025650cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.546672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 857/1715] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:21.549974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09514f-49e2-403a-bd12-1da3ded82e3f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.550124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09514f-49e2-403a-bd12-1da3ded82e3f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.550390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09514f-49e2-403a-bd12-1da3ded82e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.613579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3c09514f-49e2-403a-bd12-1da3ded82e3f None None] MFA Rules not processed for user `41bc35497fe94ababe0c824a429168d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:21.618324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3c09514f-49e2-403a-bd12-1da3ded82e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.668114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 859/1716] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:21.674418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ede9-dfce-4a3f-82c5-ea579ab0ed76 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.674908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ede9-dfce-4a3f-82c5-ea579ab0ed76 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.675213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ede9-dfce-4a3f-82c5-ea579ab0ed76 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.677009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 860/1717] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 6 11:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:21.678941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57ede9-dfce-4a3f-82c5-ea579ab0ed76 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:21.721882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57ede9-dfce-4a3f-82c5-ea579ab0ed76 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:21.737818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:21.738453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.738753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.740459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.741044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:21.742735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a71df2-0749-4cb7-88ba-71c1fd787cf8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:21.778912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 861/1718] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:21.781982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.782691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.784465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.789635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 862/1719] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 6 11:50:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:21.790861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 858/1720] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:21.791762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:21.792644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f51552-1f4d-45da-8088-2b9cbe73d5b9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.794534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f51552-1f4d-45da-8088-2b9cbe73d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.794852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f51552-1f4d-45da-8088-2b9cbe73d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.799397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.827816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:21.947033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-94f51552-1f4d-45da-8088-2b9cbe73d5b9 None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:21.950496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05e643-63c6-4190-8785-f11dc746b805 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:21.951666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-94f51552-1f4d-45da-8088-2b9cbe73d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:21.960101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:21.960744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:21.961025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:21.961299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:21.984545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:21.986117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:22.024199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 863/1721] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:22.027079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:22.075262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c95c16-93c5-4079-a2f4-8e622c0223ca service glance] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:22.102821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:22.106932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.107266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.108886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] PATH_INFO: `/v3/projects/710a9ce07e684bba8e8fa2e782c0d4c3/users/6e8ef460df3e44728114e614decabf88/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.137712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 859/1722] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:22.138684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bef8-0496-473f-8f99-8c1213a5bc3c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.138989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bef8-0496-473f-8f99-8c1213a5bc3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.139524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bef8-0496-473f-8f99-8c1213a5bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.152580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=710a9ce07e684bba8e8fa2e782c0d4c3, user_id=6e8ef460df3e44728114e614decabf88, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:22.156731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:22.171220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bc21bef8-0496-473f-8f99-8c1213a5bc3c None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:22.172894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bc21bef8-0496-473f-8f99-8c1213a5bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.188506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 864/1723] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:22 2022] PUT /identity/v3/projects/710a9ce07e684bba8e8fa2e782c0d4c3/users/6e8ef460df3e44728114e614decabf88/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:22.190542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:22.199520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.212613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:22.354493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 860/1724] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:22.356694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc080-22ec-4570-a414-843843b8221a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.357151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc080-22ec-4570-a414-843843b8221a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.357517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc080-22ec-4570-a414-843843b8221a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.383297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58d21-f199-4f8b-8371-538d830f71b4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:22.408746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:22.409434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.411279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.411601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] PATH_INFO: `/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.417211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d5e7291c514b79bbbbcc95b5e13d07)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:22.422296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-62ebc080-22ec-4570-a414-843843b8221a None None] MFA Rules not processed for user `6e8ef460df3e44728114e614decabf88`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:22.422442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:22.424438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-62ebc080-22ec-4570-a414-843843b8221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.517243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d5e7291c514b79bbbbcc95b5e13d07', 'request_id': 'req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:22.525370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d5e7291c514b79bbbbcc95b5e13d07', 'request_id': 'req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:22.536703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d5e7291c514b79bbbbcc95b5e13d07', 'request_id': 'req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:22.569474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] The token cache is being invalidate because project d9d5e7291c514b79bbbbcc95b5e13d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:22.570135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:22.571720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:22.574354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 865/1725] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:22 2022] DELETE /identity/v3/projects/d9d5e7291c514b79bbbbcc95b5e13d07 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:22.575662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:22.578267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.595981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 861/1726] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:22.598927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebc080-22ec-4570-a414-843843b8221a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:22.603677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-62ebc080-22ec-4570-a414-843843b8221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.607723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:22.700571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db7f2f-b32c-4924-a42d-fa20de7dfcc3 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:22.709880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:22.710524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.710807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.711081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.716218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.736772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:22.738278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174249f8-1396-4106-b84a-0bed6dc706b7 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:22.743477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebc080-22ec-4570-a414-843843b8221a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:22.760363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 866/1727] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:22.763837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.764369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.764654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.811721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:22.813312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.841363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:22.842043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:22.842299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:22.842577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:22.846307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:22.866949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:22.977194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 867/1728] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:22.978351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:22.983403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:23.008000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a3b19-d5df-425f-a472-0922a1de423a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.031679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.032452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.032830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.033198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.035580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.036028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.038482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.040276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e0392-6a7f-44e2-943a-c57448ee2a96 service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.072371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 862/1729] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:23.077317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.080913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.081386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.123988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 868/1730] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:22 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:23.130931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.168019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:23.174041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:23.328660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 863/1731] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:23.333017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.353294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144191e-1148-4e27-9b5b-5258946228ce None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.367723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf218a-e3cc-499f-9c38-1a996bc1aacc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.371541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.372966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.373251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.373527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.373825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.374348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:23.374452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.374602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.374748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.374999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.375664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.376244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.377602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.416369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 864/1732] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:23.420466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.424107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 869/1733] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:23 2022] POST /identity/v3/users => generated 291 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:50:23.429167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.439975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b9bb8-13a7-40ae-a48a-5c990b70d3fa admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.456759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.457393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.457695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.458030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.459293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.461466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.467159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265ea9e-aba8-45b1-8fdc-f9aea33a23e9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.486328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.489220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.489599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.492845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.494121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.499717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.536798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 870/1734] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:23.538877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 865/1735] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:23 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:50:23.543683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.543850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.559356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a065d-0ed1-4835-8de9-a27ca4bf55c4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.578640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:23.643673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.644642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.644997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.645286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] PATH_INFO: `/v3/projects/da8bc88bcb5649c8ae622adfc7966717/users/500737d918414e7bb4942b21b56ade1c/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.674907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8bc88bcb5649c8ae622adfc7966717, user_id=500737d918414e7bb4942b21b56ade1c, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.681609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.722567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f43dc-5a1a-4d8d-82d3-9deb74672959 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.732686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:23.737950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 871/1736] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:23 2022] PUT /identity/v3/projects/da8bc88bcb5649c8ae622adfc7966717/users/500737d918414e7bb4942b21b56ade1c/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:23.744110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:23.902021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.902798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.903125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.903398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.904262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.906531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.918919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a1e0-3bcb-4c18-95d9-bc31e242c62c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:23.930319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:23.931082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.931237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.931510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.932178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:23.933878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc46c6-a506-435b-817b-8acd321bee5c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:23.957584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 872/1737] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:23.959791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a99b08-a655-42ad-9c56-4800d32c7d23 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:23.960108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a99b08-a655-42ad-9c56-4800d32c7d23 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:23.960407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a99b08-a655-42ad-9c56-4800d32c7d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:23.974658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 866/1738] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:23 2022] POST /identity/v3/users => generated 292 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:23.977604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:24.012384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206d02-435c-4c22-aced-7a014e0de2e8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:24.013017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b4a99b08-a655-42ad-9c56-4800d32c7d23 None None] MFA Rules not processed for user `e05967546ef245d2b88874f6cf3f84fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:24.018388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b4a99b08-a655-42ad-9c56-4800d32c7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.028313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:24.029223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.032766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.033055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] PATH_INFO: `/v3/projects/da8bc88bcb5649c8ae622adfc7966717/users/500737d918414e7bb4942b21b56ade1c/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.044297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8bc88bcb5649c8ae622adfc7966717, user_id=500737d918414e7bb4942b21b56ade1c, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:24.061880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:24.095389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 867/1739] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:23 2022] PUT /identity/v3/projects/da8bc88bcb5649c8ae622adfc7966717/users/500737d918414e7bb4942b21b56ade1c/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:24.097063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:24.165586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 873/1740] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:24.167183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afabb0c-c937-4597-b3f7-3ad3b14ee42e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.167611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afabb0c-c937-4597-b3f7-3ad3b14ee42e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.167886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afabb0c-c937-4597-b3f7-3ad3b14ee42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.284558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6049-4987-4bff-a061-ce9796d8f146 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:24.313803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:24.314892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.316297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.316715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.317399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:24.321925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc524-d177-470e-a1fd-5d3b94dd433e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:24.343862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 868/1741] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:24.348214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.348504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.348782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.354698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-9afabb0c-c937-4597-b3f7-3ad3b14ee42e None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:24.356804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9afabb0c-c937-4597-b3f7-3ad3b14ee42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.372799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:24.427259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] MFA Rules not processed for user `500737d918414e7bb4942b21b56ade1c`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:24.431055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.499225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 874/1742] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:24.502498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.502994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.503672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.525209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 875/1743] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 6 11:50:24 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:24.526012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:24.528057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.639026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 869/1744] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:24.640284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:24.683577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d26398-fb29-431d-8237-e43264a0540e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:24.698888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:24.699841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.700254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.700668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.703159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:24.708212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:24.712604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:24.717058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28238-f352-4a4e-b05d-240e2ac200f4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:24.723945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:24.725009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.725339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.725742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.730610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.745674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 870/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:24.746706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:24.754128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.863778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:24.866271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e80b4-ac58-4553-b058-32b90a1030b1 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:24.877331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 876/1746] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:24.878761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250366-9e8f-4392-bc38-7931588224ba None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.879100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250366-9e8f-4392-bc38-7931588224ba None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.879300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250366-9e8f-4392-bc38-7931588224ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.900050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1328a-964c-41fd-a12a-ed546a6feb10 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:24.908845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-29250366-9e8f-4392-bc38-7931588224ba None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:24.910082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-29250366-9e8f-4392-bc38-7931588224ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.914020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:24.914648 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:24.914926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:24.915201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:24.918062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:24.961577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:24.963139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:24.983235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 871/1747] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:24.987707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:25.015375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:25.025511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 877/1748] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:25.025984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:25.028973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250366-9e8f-4392-bc38-7931588224ba None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:25.060007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250366-9e8f-4392-bc38-7931588224ba None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:25.075464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:25.077458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.077773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.078051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] PATH_INFO: `/v3/projects/86435fb8b2a4434ab07255d3447469d8/users/83d0ceab2af64e6ea1472db97baa572c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.115931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86435fb8b2a4434ab07255d3447469d8, user_id=83d0ceab2af64e6ea1472db97baa572c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:25.120260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:25.147427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 878/1749] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:25 2022] PUT /identity/v3/projects/86435fb8b2a4434ab07255d3447469d8/users/83d0ceab2af64e6ea1472db97baa572c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:25.149091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:25.163457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:25.179481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31aac2-27f3-4987-9cea-181c2ce54524 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:25.320096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:25.320721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.320999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.321268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.364508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da89c34-42e6-42ad-8017-51e0edf200b9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:25.378357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:25.378983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.379282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.379621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.452759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:25.468793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:25.470872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f50315-4d44-48d4-9825-fb9654ce9c8e service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:25.503855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:25.507310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:25.513694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 872/1750] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:25.525553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 879/1751] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:50:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:50:25.527924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef490f5-7c88-42e5-abcb-8d99af115dd4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.528226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef490f5-7c88-42e5-abcb-8d99af115dd4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.528503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef490f5-7c88-42e5-abcb-8d99af115dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.563344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:25.566752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:25.621889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3ef490f5-7c88-42e5-abcb-8d99af115dd4 None None] MFA Rules not processed for user `ccbb0b2070694235bf87a7b9a0c84dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:25.631985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3ef490f5-7c88-42e5-abcb-8d99af115dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:25.834167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 873/1752] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:25.840733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.841872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.841872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.920636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d7ded-3be8-4656-8371-019fdca33741 service swift] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:25.932957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:25.933601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:25.933903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:25.934173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:25.951518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:25.953402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:25.957157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] MFA Rules not processed for user `83d0ceab2af64e6ea1472db97baa572c`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:25.957917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:25.962513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:25.969847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 880/1753] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:25.974803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:25.984312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:26.014535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395551-0027-4bcb-a181-b6db6ebeba7f service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:26.025398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:26.026164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:26.026509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:26.026864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:26.031039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:26.196058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:26.197582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:26.232505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 881/1754] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:26.414362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:26.419636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 874/1755] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:26.715698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607b1b-1290-4ea9-bffd-f7f26fdf554c service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:26.738257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:26.738877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:26.739162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:26.739434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:26.745940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:26.752638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:26.851721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 882/1756] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:26 2022] POST /identity/v3/projects => generated 381 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:26.852897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:26.878581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c220d-8874-4810-80f1-2d303ce7e303 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:27.033874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:27.039475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:27.040040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:27.040959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:27.042962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:27.045526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:27.129061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:27.131447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 875/1757] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:26 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:27.152859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a2361-147c-4351-a743-6e73c46d7f62 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:27.174584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:27.175232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:27.175517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:27.175792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:27.176642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:27.190723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:27.221729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 883/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:27.225455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:27.302481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a74b8-699b-42ec-b7a7-787e4f46a340 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:27.328020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:27.329145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:27.329488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:27.329800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] PATH_INFO: `/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2/users/910c1eacc90744eca7f997c826bf5656/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:27.378320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:27.393780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc297368f32a40f78b2dea1a9f22b9a2, user_id=910c1eacc90744eca7f997c826bf5656, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:27.460522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:27.512289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 876/1759] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:27 2022] PUT /identity/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2/users/910c1eacc90744eca7f997c826bf5656/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:27.515759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:27.544236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101e368-58d5-4e16-97b6-a3a785835b4e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:27.987190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:27.988919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:27.990472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:27.991700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:27.997433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:28.038423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e80a-a3c6-4675-8b93-da1ac03064fa admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:28.094729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:28.098138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 884/1760] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 583 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:50:28.131255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11289606-8c81-4168-ad5c-d04e47c563de admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:28.164888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:28.165592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:28.170487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:28.170989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] PATH_INFO: `/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2/users/910c1eacc90744eca7f997c826bf5656/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:28.183486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc297368f32a40f78b2dea1a9f22b9a2, user_id=910c1eacc90744eca7f997c826bf5656, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:28.197319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:28.240511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 877/1761] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:28 2022] PUT /identity/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2/users/910c1eacc90744eca7f997c826bf5656/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:28.251769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:28.254638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:28.255076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:28.291466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:28.360374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] MFA Rules not processed for user `910c1eacc90744eca7f997c826bf5656`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:28.362028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:28.665890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 885/1762] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:28.673634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:28.950570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb299-ba8f-4c27-856c-ba4eb157666a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:28.965475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:28.966909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:28.967277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:28.968501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:29.024355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:29.029078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd046d-cf8f-4c87-8371-4c7557f679fb service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:29.093861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 878/1763] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:29.369692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:29.383875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c78bd-26ea-40d3-9c23-fc2d849e8ca5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:29.397790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:29.400022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:29.400888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:29.401843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:29.530614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:29.531180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:29.532812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:29.620994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:29.623046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959bf77-60ca-4fc9-af2c-e8f1ea24a726 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:29.639878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] MFA Rules not processed for user `41bc35497fe94ababe0c824a429168d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:29.647818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:29.649127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 886/1764] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:29.923245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:29.923684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:29.924050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:29.947224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 879/1765] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:30.008520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:30.017043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:30.288625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 887/1766] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:50:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:30.308683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:30.568960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711537-eca3-4245-ade2-e2642dd696fc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:30.594326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:30.597484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:30.598639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:30.599762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:30.627241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:30.631999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc390b-64ae-4b67-86df-94266671336c service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:30.657018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 880/1767] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:50:39.959992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:40.004021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:40.032851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb46549-97f8-4509-9694-af47f3d3dfa5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:40.037350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:40.038032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:40.038310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:40.038581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] PATH_INFO: `/v3/users/83d0ceab2af64e6ea1472db97baa572c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:40.039701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d0ceab2af64e6ea1472db97baa572c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:40.041598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:40.097283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d0ceab2af64e6ea1472db97baa572c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:40.108504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d0ceab2af64e6ea1472db97baa572c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:40.133316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d0ceab2af64e6ea1472db97baa572c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:40.134360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d0ceab2af64e6ea1472db97baa572c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:40.144741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d0ceab2af64e6ea1472db97baa572c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:40.152226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 888/1768] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:39 2022] DELETE /identity/v3/users/83d0ceab2af64e6ea1472db97baa572c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:40.156868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:40.157883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:40.158165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:40.180505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:40.182020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:40.192902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:40.252899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 881/1769] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:40.260813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:40.262289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:40.363049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede2a93-caa9-4e6e-a7fb-4582aed37ae7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:40.367531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:40.368226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:40.368573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:40.368942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:40.378829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:40.380416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:40.391889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 889/1770] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:40.801281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:40.806050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:40.865045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203357d7-7ec3-404c-a4c5-51ab6f319b3a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:40.882650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:40.883552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:40.886361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:40.886789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] PATH_INFO: `/v3/projects/86435fb8b2a4434ab07255d3447469d8` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:40.890293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86435fb8b2a4434ab07255d3447469d8)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:40.897060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:41.011341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86435fb8b2a4434ab07255d3447469d8', 'request_id': 'req-73f6b5c8-f698-4886-85df-72c73e593d6b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.023073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86435fb8b2a4434ab07255d3447469d8', 'request_id': 'req-73f6b5c8-f698-4886-85df-72c73e593d6b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.047152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86435fb8b2a4434ab07255d3447469d8', 'request_id': 'req-73f6b5c8-f698-4886-85df-72c73e593d6b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.073275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] The token cache is being invalidate because project 86435fb8b2a4434ab07255d3447469d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:41.073812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.074689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.077694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 882/1771] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:40 2022] DELETE /identity/v3/projects/86435fb8b2a4434ab07255d3447469d8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:41.080801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:41.091413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b259-8dd2-4488-9776-6903b37587a3 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:41.159924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:41.167989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:41.169154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:41.169841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:41.170321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] PATH_INFO: `/v3/users/910c1eacc90744eca7f997c826bf5656` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:41.172617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] RBAC: Authorizing `identity:delete_user(user_id=910c1eacc90744eca7f997c826bf5656)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:41.174808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:41.237162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910c1eacc90744eca7f997c826bf5656'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.251918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910c1eacc90744eca7f997c826bf5656'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.272158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910c1eacc90744eca7f997c826bf5656'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.273638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910c1eacc90744eca7f997c826bf5656'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.283034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8fb221e4-91c9-4344-8333-c2c0bee4209e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910c1eacc90744eca7f997c826bf5656'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:41.296369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 890/1772] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:41 2022] DELETE /identity/v3/users/910c1eacc90744eca7f997c826bf5656 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:41.784996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:41.786504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:41.811095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b5c8-f698-4886-85df-72c73e593d6b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:41.914769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:41.915470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:41.915785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:41.916052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] PATH_INFO: `/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:41.917246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc297368f32a40f78b2dea1a9f22b9a2)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:41.919611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:41.969619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:41.991271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc297368f32a40f78b2dea1a9f22b9a2', 'request_id': 'req-a6f78821-5c87-49c6-927f-3b18e720978d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:42.001386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc297368f32a40f78b2dea1a9f22b9a2', 'request_id': 'req-a6f78821-5c87-49c6-927f-3b18e720978d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:42.009550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc297368f32a40f78b2dea1a9f22b9a2', 'request_id': 'req-a6f78821-5c87-49c6-927f-3b18e720978d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:42.023174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] The token cache is being invalidate because project bc297368f32a40f78b2dea1a9f22b9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:42.023541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:42.024288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a6f78821-5c87-49c6-927f-3b18e720978d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:42.026082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 883/1773] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:41 2022] DELETE /identity/v3/projects/bc297368f32a40f78b2dea1a9f22b9a2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:42.030815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.031107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.031384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.060984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:42.062304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:42.161381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 891/1774] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:42.169818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.170421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.170702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.193037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:42.194480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:42.207199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 884/1775] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:42.212830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.232652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f7d7-1daf-4dd6-880a-4d4f44bffd29 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.239774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.241177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.241536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.241907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.242732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:42.243119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.244893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.261166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 892/1776] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:50:42.268259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.278643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d438b00-b4e7-4e9d-bfbb-6071ad170c35 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.283358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.284061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.284314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.284578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.285202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.287201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.317421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 885/1777] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:42.322500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.333426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0a4b-97ef-40b5-a2df-3f4b13146571 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.418225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.418953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.419431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.419851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.420694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.422794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.436565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:42.453175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 893/1778] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:50:42.454195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.464930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e39543-759c-4b93-9745-0e4db4086506 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.469629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.470277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.470554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.470824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.471477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.473194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.488136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 886/1779] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:42.493326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.505234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47fa9b-f3ee-416b-8cb5-b479b418ea8a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.512241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.513164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.513525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.513891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.514647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.516631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b05add-8ef3-43d4-8967-0f94c5bfef77 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.531136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 894/1780] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:42.535363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.546394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0053619-5844-42de-8550-76f2bacb1762 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.551305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.551982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.552258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.552773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] PATH_INFO: `/v3/projects/035161a9b7be44ee811c59493f4f3778/users/7bbe51b03b1347be9fcfe339065ca379/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.568835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035161a9b7be44ee811c59493f4f3778, user_id=7bbe51b03b1347be9fcfe339065ca379, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.571543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.586004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 887/1781] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:42 2022] PUT /identity/v3/projects/035161a9b7be44ee811c59493f4f3778/users/7bbe51b03b1347be9fcfe339065ca379/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:42.597449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.598617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.598617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.629735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] MFA Rules not processed for user `7bbe51b03b1347be9fcfe339065ca379`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:42.631943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:42.725897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 895/1782] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:42.730397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.812133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387eb27e-6cab-4128-9229-769a26293c45 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.818115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.819536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.819885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.820230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.821703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.823303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:42.869707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:42.886106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 888/1783] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:42.890243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100599-eb47-4ab3-9ad2-b1e6fca50457 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:42.981074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:42.982119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:42.982538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:42.982891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:42.984426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:42.986988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92997c-5168-4334-a0cc-755748e030a4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:43.032853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 896/1784] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:42 2022] POST /identity/v3/users => generated 301 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:43.038043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.060220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d52e-43ce-4f2d-8cae-a39f2888bc4e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:43.063959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.063959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.064330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.082640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:43.086583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.087893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.088370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.091625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:43.097713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:43.119261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 889/1785] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:43.127660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.154671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:43.155897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:43.166253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3bea-b475-4e3b-b2c5-610ecea54eba admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:43.183529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:43.184488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.189044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.201265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] PATH_INFO: `/v3/projects/5b0b536e05a6443ca14c2077c4340e2e/users/4451be4372e44ff78936d6055287c3a3/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.223983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0b536e05a6443ca14c2077c4340e2e, user_id=4451be4372e44ff78936d6055287c3a3, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:43.229115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:43.299075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 890/1786] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:43 2022] PUT /identity/v3/projects/5b0b536e05a6443ca14c2077c4340e2e/users/4451be4372e44ff78936d6055287c3a3/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:43.306580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.401715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:43.449284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 897/1787] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Sep 6 11:50:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:43.586847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02b34-5146-4d11-9c0f-6c798d0bab72 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:43.612128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:43.612767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.613041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.613309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.613999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:43.616839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e588b-c47c-42bd-bf8e-c0e38f2b86e0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:43.627626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 891/1788] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:43.632871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.645870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc06c-e8b0-45f9-9d18-6841cb353399 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:43.651758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:43.652433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.652707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.653126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] PATH_INFO: `/v3/projects/5b0b536e05a6443ca14c2077c4340e2e/users/4451be4372e44ff78936d6055287c3a3/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.655081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0b536e05a6443ca14c2077c4340e2e, user_id=4451be4372e44ff78936d6055287c3a3, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:43.657788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:43.664672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:43.670414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 898/1789] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:43 2022] PUT /identity/v3/projects/5b0b536e05a6443ca14c2077c4340e2e/users/4451be4372e44ff78936d6055287c3a3/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:43.682319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523301-812a-4889-a247-43a64b6215dd None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.682636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523301-812a-4889-a247-43a64b6215dd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.682914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523301-812a-4889-a247-43a64b6215dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:43.711815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-91523301-812a-4889-a247-43a64b6215dd None None] MFA Rules not processed for user `4451be4372e44ff78936d6055287c3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:43.713041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-91523301-812a-4889-a247-43a64b6215dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:43.782832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 892/1790] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:43.794465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.796189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:43.845112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523301-812a-4889-a247-43a64b6215dd None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:43.846812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-91523301-812a-4889-a247-43a64b6215dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:43.932175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c51b-7e3e-44ac-a76a-63bdf90f7004 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:43.940581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:43.942318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:43.942963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:43.943506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:44.008430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:44.010500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:44.011851 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523301-812a-4889-a247-43a64b6215dd None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:44.023748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:44.024575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:44.025245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:44.025636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:44.039226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 899/1791] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:50:44.054291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:44.057929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e68b6-3f12-4dc1-ae71-b96d6e3e2aec service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:44.089221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 893/1792] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:44.266958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:44.442950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a210ad-cdf5-4027-8726-94eca9873603 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:44.450913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:44.452340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:44.452862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:44.453338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] PATH_INFO: `/v3/users/7bbe51b03b1347be9fcfe339065ca379` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:44.454843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbe51b03b1347be9fcfe339065ca379)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:44.457595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:44.561357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbe51b03b1347be9fcfe339065ca379'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:44.574181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbe51b03b1347be9fcfe339065ca379'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:44.609476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbe51b03b1347be9fcfe339065ca379'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:44.610308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbe51b03b1347be9fcfe339065ca379'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:44.627422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbe51b03b1347be9fcfe339065ca379'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:44.647691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 900/1793] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:44 2022] DELETE /identity/v3/users/7bbe51b03b1347be9fcfe339065ca379 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:44.649605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:44.650156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:44.650773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:44.690162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:44.691760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:44.809998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:44.858974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 894/1794] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:44.905374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:44.906846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:44.938189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:45.017296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5578e4-562b-48d9-85ed-d0b65126cf41 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:45.021264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:45.022297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:45.022641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:45.023016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:45.034623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:45.037220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831c66d-98b4-4275-b16c-b38074e5bb19 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:45.050604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 901/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:46.777518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:46.782297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:46.812977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d898e-305c-4aaa-9653-4bdeb128b8e5 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:46.857576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:46.858354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:46.860943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:46.861231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] PATH_INFO: `/v3/projects/035161a9b7be44ee811c59493f4f3778` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:46.864776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035161a9b7be44ee811c59493f4f3778)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:46.873004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:46.972938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef4d4-1b26-4b2d-a9b2-508949a2b054 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:46.973707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef4d4-1b26-4b2d-a9b2-508949a2b054 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:46.973707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aef4d4-1b26-4b2d-a9b2-508949a2b054 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:46.975390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 902/1796] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:46.978942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035161a9b7be44ee811c59493f4f3778', 'request_id': 'req-392961c4-622d-45ab-a195-63effe5f75f1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:46.998905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035161a9b7be44ee811c59493f4f3778', 'request_id': 'req-392961c4-622d-45ab-a195-63effe5f75f1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.007999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035161a9b7be44ee811c59493f4f3778', 'request_id': 'req-392961c4-622d-45ab-a195-63effe5f75f1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.045157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:47.045743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:47.046160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:47.050893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] The token cache is being invalidate because project 035161a9b7be44ee811c59493f4f3778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:47.051486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.054768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.058324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 895/1797] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:46 2022] DELETE /identity/v3/projects/035161a9b7be44ee811c59493f4f3778 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:47.062193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:47.064239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:47.080072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:47.082344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:47.188282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 903/1798] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:50:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:47.191489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392961c4-622d-45ab-a195-63effe5f75f1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:47.198387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:47.199129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:47.199579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:47.202306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] PATH_INFO: `/v3/users/4451be4372e44ff78936d6055287c3a3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:47.204580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4451be4372e44ff78936d6055287c3a3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:47.206843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:47.281902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4451be4372e44ff78936d6055287c3a3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.282816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4451be4372e44ff78936d6055287c3a3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.285469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:47.288226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:47.305434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4451be4372e44ff78936d6055287c3a3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.310810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:47.338855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4451be4372e44ff78936d6055287c3a3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.340468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4451be4372e44ff78936d6055287c3a3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:47.373018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864fc5-7151-492a-9ff4-5924da94329f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:47.375182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 896/1799] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:47 2022] DELETE /identity/v3/users/4451be4372e44ff78936d6055287c3a3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:47.477224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:47.478051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:47.478459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:47.479558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:47.480944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:47.499261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:47.500856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23261406-bf26-484d-b650-988c8d506a5c service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:47.527112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 904/1800] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:50:49.449179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:49.450806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:49.543416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7e6a8-cd82-46ef-890d-d8ea2d161922 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:49.548847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:49.550359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:49.550359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:49.550359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] PATH_INFO: `/v3/projects/5b0b536e05a6443ca14c2077c4340e2e` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:49.553912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0b536e05a6443ca14c2077c4340e2e)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:49.587961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:49.697877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0b536e05a6443ca14c2077c4340e2e', 'request_id': 'req-854a2c66-4c7d-4769-876e-490686e19272'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:49.746695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0b536e05a6443ca14c2077c4340e2e', 'request_id': 'req-854a2c66-4c7d-4769-876e-490686e19272'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:49.757019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0b536e05a6443ca14c2077c4340e2e', 'request_id': 'req-854a2c66-4c7d-4769-876e-490686e19272'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:49.770145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] The token cache is being invalidate because project 5b0b536e05a6443ca14c2077c4340e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:49.770511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:49.772849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-854a2c66-4c7d-4769-876e-490686e19272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:49.774918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 897/1801] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:49 2022] DELETE /identity/v3/projects/5b0b536e05a6443ca14c2077c4340e2e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:49.779453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:49.780733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:49.780733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:49.801762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:49.803886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:49.857539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:49.900887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 905/1802] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:49.902105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:49.902389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:49.902660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:49.937608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:49.946761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:49.958343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 898/1803] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:49.962530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:49.976723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c249b-7f22-44d1-8062-3b46f702fe6b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:49.984986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:49.986620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:49.986906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:49.987795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:49.988407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:49.988724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:49.992054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.014627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 906/1804] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:50.015400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.028451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209dd70-8542-4d4e-acd2-618b8707dac9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.034034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.034654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.035082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.035360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.036102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.038841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.109732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 899/1805] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:50 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:50.110991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.135192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3db1-5f5a-4fd3-af53-866ef5b262eb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.221932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.222653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.223171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.223370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.225112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.226659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.253822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 907/1806] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:50 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:50.262791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.273235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a79f9-aded-4d03-a78c-f2f73e5f85e0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.278712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.279352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.279616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.279891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.280562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.282385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.297043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 900/1807] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:50.298266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.316685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963fa38-cbb7-4fe1-a9f1-abfafeb36334 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.322910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.323804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.324133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.324394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.325509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.327583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4581-a5b5-4273-aca5-6ad82cd09edc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.339751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 908/1808] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:50.344000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.356073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433dcd4-2ae2-4910-80c4-36654076b6ab admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.360824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.361491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.361834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.362111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] PATH_INFO: `/v3/projects/0b6ce6f29f474a449f63837f118f0795/users/7b969fd93ebc4e6d914464acb235deeb/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.378407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6ce6f29f474a449f63837f118f0795, user_id=7b969fd93ebc4e6d914464acb235deeb, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.381456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.397212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 901/1809] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:50 2022] PUT /identity/v3/projects/0b6ce6f29f474a449f63837f118f0795/users/7b969fd93ebc4e6d914464acb235deeb/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:50.408317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.408599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.408873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.455540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] MFA Rules not processed for user `7b969fd93ebc4e6d914464acb235deeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:50.457161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:50.557886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 909/1810] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:50:50.562782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.575469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:50.660224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43c757-287c-4fbf-a775-55f880af1942 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.673936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.674725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.675005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.675273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.675920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.678637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.732376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 902/1811] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:50 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:50:50.736519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.751125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413d24-ee2a-40cb-a208-c88445c59eb1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.795292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:50.837745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.838552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.839074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.839707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.840569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.843061 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.882587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 910/1812] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:50 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:50.886340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.904325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ce5f-c9de-4ec3-98fa-001aac1a5986 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.911297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.912999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.913389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.914107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.915080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.916881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:50.937178 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 903/1813] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:50.937883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:50.951771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529a0a1-ec10-44e1-a015-c611daefc90e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:50.959149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:50.960157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:50.960621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:50.961089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] PATH_INFO: `/v3/projects/272b4b1163624b3798649ca8f3a7307b/users/48f26a72f212436daacce650a02a343d/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:50.983109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=272b4b1163624b3798649ca8f3a7307b, user_id=48f26a72f212436daacce650a02a343d, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:50.986479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:51.015050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 911/1814] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:50 2022] PUT /identity/v3/projects/272b4b1163624b3798649ca8f3a7307b/users/48f26a72f212436daacce650a02a343d/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:51.019267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:51.035542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc722244-fe90-4e4e-b1cf-363be98e9b9e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:51.123627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:51.124267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.124547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.124825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.125826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:51.127905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd1d9d-ea85-4991-b693-d58f5e316f90 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:51.144831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 904/1815] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:51.146085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:51.158027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813528b9-1983-45b3-8920-29fd64fd2fec admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:51.163461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:51.164297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.164656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.165028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] PATH_INFO: `/v3/projects/272b4b1163624b3798649ca8f3a7307b/users/48f26a72f212436daacce650a02a343d/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.167451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272b4b1163624b3798649ca8f3a7307b, user_id=48f26a72f212436daacce650a02a343d, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:51.170505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd583f4-06d2-4096-996f-d746f2541e72 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:51.183150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 912/1816] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:51 2022] PUT /identity/v3/projects/272b4b1163624b3798649ca8f3a7307b/users/48f26a72f212436daacce650a02a343d/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:51.194286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.194839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.195187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.232992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] MFA Rules not processed for user `48f26a72f212436daacce650a02a343d`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:51.234527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:51.314062 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3c3c-2e10-44fb-b695-91914304b713 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.314608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3c3c-2e10-44fb-b695-91914304b713 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.315777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3c3c-2e10-44fb-b695-91914304b713 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.317491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 913/1817] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:50:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:51.330618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 905/1818] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:51.335144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.335510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.336303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.346252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:51.347989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:51.356739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:51.367676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:51.369415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:51.419235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516e23e-9da9-450a-8ad1-1217bd8a6f1e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:51.424155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:51.424909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.425286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.425675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.437993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:51.439841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:51.476554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 906/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:51.512266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 914/1820] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:51.605522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:51.608370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:51.641299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca186ba-b4c5-4945-ab08-3decfcb54e6c service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:51.649128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:51.649951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:51.650318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:51.650722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:51.666599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:51.668322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfe0cc-9d9b-4adc-bc00-4b6bda291557 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:51.680799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 907/1821] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:52.175158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:52.269837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569607d-2048-4c49-a05f-30e2a1e4845d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:52.276773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:52.277600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:52.277967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:52.278311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] PATH_INFO: `/v3/users/7b969fd93ebc4e6d914464acb235deeb` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:52.279721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b969fd93ebc4e6d914464acb235deeb)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:52.283393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:52.324073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:52.395821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b969fd93ebc4e6d914464acb235deeb'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:52.406073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b969fd93ebc4e6d914464acb235deeb'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:52.422901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b969fd93ebc4e6d914464acb235deeb'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:52.423784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b969fd93ebc4e6d914464acb235deeb'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:52.433823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b969fd93ebc4e6d914464acb235deeb'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:52.444630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 915/1822] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:52 2022] DELETE /identity/v3/users/7b969fd93ebc4e6d914464acb235deeb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:52.446659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:52.446948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:52.447220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:52.468882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:52.470340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:52.557361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 908/1823] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:52.620930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:52.622939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:52.792697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb67da5-5dfe-4efa-9976-a275de2b554e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:52.800067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:52.800947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:52.801402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:52.806464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:52.825721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:52.827839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:52.842159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 916/1824] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:53.987802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:53.989383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:54.024407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602b04-2faa-49e7-be6f-899c2a28dcf7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:54.041481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:54.042316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:54.042539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:54.042831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] PATH_INFO: `/v3/projects/0b6ce6f29f474a449f63837f118f0795` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:54.044157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6ce6f29f474a449f63837f118f0795)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:54.046159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:54.094404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6ce6f29f474a449f63837f118f0795', 'request_id': 'req-176825b7-df59-4cf8-858c-7668c8ac247f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.103003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6ce6f29f474a449f63837f118f0795', 'request_id': 'req-176825b7-df59-4cf8-858c-7668c8ac247f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.109098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6ce6f29f474a449f63837f118f0795', 'request_id': 'req-176825b7-df59-4cf8-858c-7668c8ac247f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.120833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] The token cache is being invalidate because project 0b6ce6f29f474a449f63837f118f0795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:54.121119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.121792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.123311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 909/1825] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:53 2022] DELETE /identity/v3/projects/0b6ce6f29f474a449f63837f118f0795 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:54.127362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:54.137837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b771-eb9c-427f-a95b-653f6bf68034 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:54.199820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:54.200467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:54.200742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:54.201011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] PATH_INFO: `/v3/users/48f26a72f212436daacce650a02a343d` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:54.202162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f26a72f212436daacce650a02a343d)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:54.204109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:54.251178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:54.254158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f26a72f212436daacce650a02a343d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.263887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f26a72f212436daacce650a02a343d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.277192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f26a72f212436daacce650a02a343d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.277916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f26a72f212436daacce650a02a343d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.284683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c1ae00be-8350-4291-90ff-0733d439a3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f26a72f212436daacce650a02a343d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.291743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 917/1826] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:54 2022] DELETE /identity/v3/users/48f26a72f212436daacce650a02a343d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:54.631846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:54.633679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:54.665295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176825b7-df59-4cf8-858c-7668c8ac247f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:54.686733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:54.773999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:54.774891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:54.775193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:54.775439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] PATH_INFO: `/v3/projects/272b4b1163624b3798649ca8f3a7307b` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:54.776650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272b4b1163624b3798649ca8f3a7307b)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:54.778759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:54.836435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272b4b1163624b3798649ca8f3a7307b', 'request_id': 'req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.845618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272b4b1163624b3798649ca8f3a7307b', 'request_id': 'req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.854861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272b4b1163624b3798649ca8f3a7307b', 'request_id': 'req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.870373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] The token cache is being invalidate because project 272b4b1163624b3798649ca8f3a7307b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:54.870723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.871436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ac4bcf8-cbbc-4af9-8398-1e3d58e9c213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:54.873194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 910/1827] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:54 2022] DELETE /identity/v3/projects/272b4b1163624b3798649ca8f3a7307b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:54.878530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:54.878915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:54.879341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:54.914904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:54.916163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:54.987198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 918/1828] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:54.997303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:54.997818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:54.997896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.022922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:55.024164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:55.033849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 911/1829] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:55.039578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.061721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5c5f-bbf0-4268-b9fa-0ee63ace5214 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.069717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.070393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.070679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.070959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.071660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:55.071966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.073637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.086859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 919/1830] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:55.091525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.103026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eba3a-4976-406b-97cd-1ffa8f81c214 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.107971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.108591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.108874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.109139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.109920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.111861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.151094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 912/1831] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:55 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:55.152381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.163311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3990c-3996-435c-87af-a5233e7ca7df admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.229310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.229970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.230248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.230513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.231169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.233093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.260372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 920/1832] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:55 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:50:55.262010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.275284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e19f7-a4bb-40ee-9d2e-e2d6546e1c66 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.280134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.280751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.281031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.281303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.281986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.283622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.295203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 913/1833] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:50:55.299294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.311296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef1273-aa15-426a-8577-456d3163b2dc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.316068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.316678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.316951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.317219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.318000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.320219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100b8c4-83ff-42a0-aac1-2eabbe4182ff admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.331572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 921/1834] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:55.336583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.346717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922abb-5178-4e94-b4a0-019cb5f15012 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.351979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.352630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.352902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.353169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] PATH_INFO: `/v3/projects/8fe088859862453fa191b3f0101f7545/users/492dc75c19a24fa480ca80801af9b87a/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.370086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe088859862453fa191b3f0101f7545, user_id=492dc75c19a24fa480ca80801af9b87a, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.372752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.387575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 914/1835] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:55 2022] PUT /identity/v3/projects/8fe088859862453fa191b3f0101f7545/users/492dc75c19a24fa480ca80801af9b87a/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:55.399670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.399879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.400183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.430079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] MFA Rules not processed for user `492dc75c19a24fa480ca80801af9b87a`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:55.431447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:55.450757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:55.516744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 922/1836] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:55.529232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.531004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:55.586630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:55.605701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2004fe6-3cf1-44dc-841a-a1d0b490edca admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.608641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.609360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.609811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.610065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.621050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.623878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbb901-f14a-494a-a91e-bcb96eca7a3c service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.640938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 915/1837] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:55.664888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:55.734974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ec84e-12ec-4a64-afe8-458e2f9d82ad None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:55.739867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:55.740245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.740523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.740797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] PATH_INFO: `/v3/users/492dc75c19a24fa480ca80801af9b87a` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.741958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=492dc75c19a24fa480ca80801af9b87a)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:55.744130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:55.797986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '492dc75c19a24fa480ca80801af9b87a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:55.808494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '492dc75c19a24fa480ca80801af9b87a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:55.827050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '492dc75c19a24fa480ca80801af9b87a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:55.827841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '492dc75c19a24fa480ca80801af9b87a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:55.837232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '492dc75c19a24fa480ca80801af9b87a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:55.850826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 923/1838] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:50:55 2022] DELETE /identity/v3/users/492dc75c19a24fa480ca80801af9b87a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:55.856391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea365c2-bfdf-42d1-81db-801afd85bdb0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:55.856683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea365c2-bfdf-42d1-81db-801afd85bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:55.856962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea365c2-bfdf-42d1-81db-801afd85bdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:55.879974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-aea365c2-bfdf-42d1-81db-801afd85bdb0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:55.881988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-aea365c2-bfdf-42d1-81db-801afd85bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:55.984263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 916/1839] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:55.998245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:56.000022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.061042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:56.091119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c64b7-48f0-452c-9085-84a8c5886bc9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:56.095676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:56.096831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.096831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.097140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.112669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:56.114402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aeb66-4063-460b-942b-eb308bd6a291 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:56.129436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 924/1840] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:56.370951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.372002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.372473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.374169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 917/1841] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:50:56.383148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.384018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.384018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.445540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:56.447030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.546667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 925/1842] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:56.578186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:56.579951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.600533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6281d79-3982-4499-8621-02882dd5d273 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:56.609146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:56.609874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.610063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.610342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.620703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:56.622343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e358d4f-ce5c-4779-890d-077e5e9f11dd service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:56.626648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:56.628332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.646233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6d082-c6e3-4061-be31-ddf652405337 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:56.654293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:56.654761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.655250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.655525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] PATH_INFO: `/v3/projects/8fe088859862453fa191b3f0101f7545` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.658409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe088859862453fa191b3f0101f7545)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:56.659566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 918/1843] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:50:56.661043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:56.718209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe088859862453fa191b3f0101f7545', 'request_id': 'req-fb44734a-d1d5-45f9-bb5d-e712799d59ad'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:56.725821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe088859862453fa191b3f0101f7545', 'request_id': 'req-fb44734a-d1d5-45f9-bb5d-e712799d59ad'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:56.734115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe088859862453fa191b3f0101f7545', 'request_id': 'req-fb44734a-d1d5-45f9-bb5d-e712799d59ad'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:56.745924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] The token cache is being invalidate because project 8fe088859862453fa191b3f0101f7545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:50:56.746200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:56.746972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-fb44734a-d1d5-45f9-bb5d-e712799d59ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:50:56.748485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 926/1844] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:50:56 2022] DELETE /identity/v3/projects/8fe088859862453fa191b3f0101f7545 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:50:56.753554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.753890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.754142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.782928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:56.784279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.852065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 919/1845] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:56.859763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.860052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.860331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.881395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:56.882670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:56.890916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 927/1846] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:56.895191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:56.905286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bc205-e5c3-4a00-a377-adee0803c386 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:56.910042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:56.910640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.910916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.911221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.912506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:50:56.912956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:56.914640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:56.932593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 920/1847] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:50:56.933805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:56.946954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13740d7e-6187-4cf4-829f-a36fbfd2c0e5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:56.951975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:56.952649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:56.952935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:56.953241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:56.953921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:56.956072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:56.984608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 928/1848] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:50:56 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:56.989063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:57.000693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08131-59e4-4a3f-955b-d4bc933b7aca admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:57.066664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:57.067866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.069017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.069483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.070219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:57.072546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:57.091310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:57.117581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 921/1849] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:50:56 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:50:57.123685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:57.141277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c32a-a97a-4123-b662-2260ae987606 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:57.150313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:57.151520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.152233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.152851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.154402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:57.156934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:57.170921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 929/1850] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:50:57.174991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:57.198672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d2c4f-06ca-497e-b40c-4a569cdf67df admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:57.207584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:57.208208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.209005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.209278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.209972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:57.211592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e904a8-daa2-45cd-82de-04a049aaad80 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:57.229580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 922/1851] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:50:57.235544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:57.248860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791b87-f1e6-4d71-87ef-cb68ba6b479b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:57.254392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:57.255074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.255652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.255947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] PATH_INFO: `/v3/projects/2b23b9bb54c145729b7f1b74e5ea0fe5/users/a6a8b8da9a464b9e8d5e15fd10b3b28c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.268853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:50:57.274162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b23b9bb54c145729b7f1b74e5ea0fe5, user_id=a6a8b8da9a464b9e8d5e15fd10b3b28c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:57.276925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:57.292271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 930/1852] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:50:57 2022] PUT /identity/v3/projects/2b23b9bb54c145729b7f1b74e5ea0fe5/users/a6a8b8da9a464b9e8d5e15fd10b3b28c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:50:57.302425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.303092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.303519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.340401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] MFA Rules not processed for user `a6a8b8da9a464b9e8d5e15fd10b3b28c`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:50:57.341712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:57.461927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 923/1853] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:50:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:50:57.471809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:50:57.474523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:50:57.570898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cfa0-303f-4d6e-a086-3d214dbabfcb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:50:57.574928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:50:57.575678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:50:57.576028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:50:57.576372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:50:57.594599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:50:57.594599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b296e7-55dd-485f-b6a5-9cc86b1892ab service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:50:57.597496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 931/1854] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:50:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:07.454075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:07.593583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:16.825847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:16.897160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c2680-aee5-4739-8dbc-c04a93403b3c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:16.903586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:16.903903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:16.904759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:16.905005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] PATH_INFO: `/v3/users/a6a8b8da9a464b9e8d5e15fd10b3b28c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:16.906497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a8b8da9a464b9e8d5e15fd10b3b28c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:16.908429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:16.957528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a8b8da9a464b9e8d5e15fd10b3b28c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:16.958318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a8b8da9a464b9e8d5e15fd10b3b28c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:16.973125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a8b8da9a464b9e8d5e15fd10b3b28c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:16.984353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a8b8da9a464b9e8d5e15fd10b3b28c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:16.985045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a8b8da9a464b9e8d5e15fd10b3b28c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.010916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 924/1855] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:16 2022] DELETE /identity/v3/users/a6a8b8da9a464b9e8d5e15fd10b3b28c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:17.012581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:17.013181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:17.013181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:17.034767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:17.036151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:17.147229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 932/1856] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:17.178093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:17.181092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:17.299373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1fd4d-fe45-411d-be0c-da4efeb704c3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:17.303816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:17.305308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:17.305599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:17.305910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:17.318304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:17.320618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873096-ed34-45fd-903d-bb7feb481fa7 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:17.333292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 925/1857] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:17.706658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:17.708626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:17.721138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184fd4-4539-4dec-a3dc-9dc38b230194 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:17.726143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:17.726882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:17.727184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:17.727416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] PATH_INFO: `/v3/projects/2b23b9bb54c145729b7f1b74e5ea0fe5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:17.728562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b23b9bb54c145729b7f1b74e5ea0fe5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:17.730798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:17.796838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b23b9bb54c145729b7f1b74e5ea0fe5', 'request_id': 'req-4447ec48-32b3-481b-ae07-241d2b95f0f8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.806342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b23b9bb54c145729b7f1b74e5ea0fe5', 'request_id': 'req-4447ec48-32b3-481b-ae07-241d2b95f0f8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.816772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b23b9bb54c145729b7f1b74e5ea0fe5', 'request_id': 'req-4447ec48-32b3-481b-ae07-241d2b95f0f8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.831720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] The token cache is being invalidate because project 2b23b9bb54c145729b7f1b74e5ea0fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:17.832254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.833505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4447ec48-32b3-481b-ae07-241d2b95f0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:17.844229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 933/1858] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:17 2022] DELETE /identity/v3/projects/2b23b9bb54c145729b7f1b74e5ea0fe5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:17.849063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef156-7305-45ab-889d-bee93d2363be None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:17.849605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef156-7305-45ab-889d-bee93d2363be None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:17.849605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef156-7305-45ab-889d-bee93d2363be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:17.875824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cacef156-7305-45ab-889d-bee93d2363be None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:17.877060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cacef156-7305-45ab-889d-bee93d2363be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:17.900262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:17.978603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 926/1859] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:17.984767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:17.985053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:17.985331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.012052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:18.013276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:18.036657 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 934/1860] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:18.042069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef156-7305-45ab-889d-bee93d2363be None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.056086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef156-7305-45ab-889d-bee93d2363be None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.061552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.062208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.062536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.062752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.063416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:51:18.063722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.065445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.082376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 927/1861] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:51:18.084056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.093918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9a707-26ee-4269-8bfa-78cc89e72716 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.098624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.099229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.099506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.099774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.100447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.102379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.129723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 935/1862] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:18 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:18.134578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.145172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36abc1a-a542-4357-9ecb-e69479c90b85 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.201064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.201701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.202053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.202331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.202987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.204968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.230041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 928/1863] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:18 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:51:18.232105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.243121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df91b8f-2fc4-49b6-89c0-c0eb3e0f3ae5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.248431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.249065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.249342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.250078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.250371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.252011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.267341 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 936/1864] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:18.274919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.284738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01cac6-4513-4987-b075-b0372542bbc9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.290063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.290766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.292753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.292753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.292753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.293850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74f5f9-74a2-4466-bf6d-84100b5e58ed admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.315804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 929/1865] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:18.320931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.333670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5213-007c-4cbb-b001-98e88ef1a362 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.339099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.339778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.340114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.340331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] PATH_INFO: `/v3/projects/24b14fcbaeb647e79ec038c4c5cfe013/users/eca27d28e19a434eae7195f4061bff44/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.360364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b14fcbaeb647e79ec038c4c5cfe013, user_id=eca27d28e19a434eae7195f4061bff44, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.363427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.388993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 937/1866] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:18 2022] PUT /identity/v3/projects/24b14fcbaeb647e79ec038c4c5cfe013/users/eca27d28e19a434eae7195f4061bff44/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:18.399144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bb4a-1c72-4577-bd38-4b36b724076a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.399567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bb4a-1c72-4577-bd38-4b36b724076a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.399944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bb4a-1c72-4577-bd38-4b36b724076a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.440323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-12b7bb4a-1c72-4577-bd38-4b36b724076a None None] MFA Rules not processed for user `eca27d28e19a434eae7195f4061bff44`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:18.441590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-12b7bb4a-1c72-4577-bd38-4b36b724076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:18.540385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 930/1867] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:18.552267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:18.558722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:18.582822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:18.626453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0b31-ed56-47f6-8ef3-2bc536a3d826 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:18.630673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:18.631347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:18.631707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:18.632050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:18.642899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:18.644545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:18.662352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 938/1868] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:20.148738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:20.149363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:20.149475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:20.187733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:20.188539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:20.223041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:20.363294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 931/1869] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:20.379050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:20.381619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:20.531509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac46bf-d88a-4f6b-a451-c9e3f447b836 service glance] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:20.537325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:20.538298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:20.538962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:20.538962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:20.549286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:20.551189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996793d-0131-4969-bd51-ef825f561e80 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:20.561218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 939/1870] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:51:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:51:30.346212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=17 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:30.557610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=49 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:39.842452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:39.843988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:39.946053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7ad1c-d376-41dc-b258-8a42a15284a9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:39.952565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:39.954192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:39.954633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:39.955018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] PATH_INFO: `/v3/users/6e8ef460df3e44728114e614decabf88` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:39.956637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8ef460df3e44728114e614decabf88)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:39.960913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:40.061758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8ef460df3e44728114e614decabf88'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.062529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8ef460df3e44728114e614decabf88'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.077686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8ef460df3e44728114e614decabf88'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.088684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8ef460df3e44728114e614decabf88'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.089357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8ef460df3e44728114e614decabf88'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.107724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 932/1871] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:39 2022] DELETE /identity/v3/users/6e8ef460df3e44728114e614decabf88 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:40.112267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:40.112829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:40.112829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:40.134969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:40.136395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:40.225764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 940/1872] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:40.238553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:40.240007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:40.305091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56f7b7-46b6-48d9-ad41-108fea2ff3af admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:40.308834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:40.309490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:40.309746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:40.310029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:40.320253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:40.321724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:40.331495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 933/1873] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:40.871254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:40.873818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:40.888238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef02cb-14e7-4cd3-9891-5e06bafdf7e9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:40.898130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:40.898787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:40.899071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:40.899340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] PATH_INFO: `/v3/projects/710a9ce07e684bba8e8fa2e782c0d4c3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:40.900509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=710a9ce07e684bba8e8fa2e782c0d4c3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:40.902681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:40.966259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '710a9ce07e684bba8e8fa2e782c0d4c3', 'request_id': 'req-cd18d5e5-2f98-47e1-9560-0f912440f278'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.973065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '710a9ce07e684bba8e8fa2e782c0d4c3', 'request_id': 'req-cd18d5e5-2f98-47e1-9560-0f912440f278'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.985676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '710a9ce07e684bba8e8fa2e782c0d4c3', 'request_id': 'req-cd18d5e5-2f98-47e1-9560-0f912440f278'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.996583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] The token cache is being invalidate because project 710a9ce07e684bba8e8fa2e782c0d4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:40.997285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.998009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:40.999708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 941/1874] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:40 2022] DELETE /identity/v3/projects/710a9ce07e684bba8e8fa2e782c0d4c3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:41.003762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:41.019531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113fd41-d596-41e9-8f28-bcc496f6c13e service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:41.039758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:41.088692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:41.090213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.090213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.090213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] PATH_INFO: `/v3/users/500737d918414e7bb4942b21b56ade1c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.091244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500737d918414e7bb4942b21b56ade1c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:41.093197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:41.166813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500737d918414e7bb4942b21b56ade1c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.168563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500737d918414e7bb4942b21b56ade1c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.192791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500737d918414e7bb4942b21b56ade1c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.208858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500737d918414e7bb4942b21b56ade1c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.211246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d2bc8561-4ee9-491a-b3ab-2206b41e4107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500737d918414e7bb4942b21b56ade1c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.227358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 934/1875] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:41 2022] DELETE /identity/v3/users/500737d918414e7bb4942b21b56ade1c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:41.600092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:41.601800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:41.616701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18d5e5-2f98-47e1-9560-0f912440f278 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:41.679252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:41.679924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.680206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.680477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] PATH_INFO: `/v3/projects/da8bc88bcb5649c8ae622adfc7966717` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.681623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8bc88bcb5649c8ae622adfc7966717)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:41.683571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:41.720504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:41.741908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8bc88bcb5649c8ae622adfc7966717', 'request_id': 'req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.747710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8bc88bcb5649c8ae622adfc7966717', 'request_id': 'req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.755365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8bc88bcb5649c8ae622adfc7966717', 'request_id': 'req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.767860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] The token cache is being invalidate because project da8bc88bcb5649c8ae622adfc7966717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:41.768162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.768757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-a70ddfb2-2efa-46ff-9060-9cdda34b64ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:41.770550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 942/1876] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:41 2022] DELETE /identity/v3/projects/da8bc88bcb5649c8ae622adfc7966717 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:41.776570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.776860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.777137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.800345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:41.801584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:41.883188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 935/1877] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:41.891594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.891924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.892143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.911743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:41.913710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:41.921841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 943/1878] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:51:41.926081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:41.936789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f5d8c-f55a-44fa-94f9-6c4c442840a2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:41.942130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:41.942827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.943099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.943458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.944247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:51:41.944496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:41.946140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:41.958597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 936/1879] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:51:41.964265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:41.973630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb5d5-4f38-4b6f-a929-fbd17d90a95d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:41.979616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:41.980217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:41.980497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:41.980770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:41.981414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:41.983364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.024140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 944/1880] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:41 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:42.029364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.044451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639e74-d72b-4a24-9b2b-03ab4f093b5d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.076829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:42.127223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.127923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.132658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.133084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.133865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.135818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.162207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 937/1881] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:42 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:42.167366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.178258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40f8af-ddfe-4028-9d78-9a401f672821 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.183821 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.184558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.184888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.185164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.185844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.187453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.198078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 945/1882] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:51:42.203252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.214811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ec32c-3706-4938-81a3-bddcebd09cb1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.220167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.220819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.221170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.221362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.222158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.223708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d0e66-0e5b-4488-91cc-9b486a5247e6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.239974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 938/1883] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:51:42.248713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.271636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0da030-6b2f-4af9-b043-7fa4e419928a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.276793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.277396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.277703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.277983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] PATH_INFO: `/v3/projects/80e764accca34094b0b320af8515451e/users/9c8ccfedb13349b68490227921cd2c8b/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.311068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e764accca34094b0b320af8515451e, user_id=9c8ccfedb13349b68490227921cd2c8b, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.314004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.353292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 946/1884] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:42 2022] PUT /identity/v3/projects/80e764accca34094b0b320af8515451e/users/9c8ccfedb13349b68490227921cd2c8b/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:42.354787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.355068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.355407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.398144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] MFA Rules not processed for user `9c8ccfedb13349b68490227921cd2c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:42.399397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:42.514002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 939/1885] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:42.515242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.615072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae86c7-409a-43f4-8226-b701abe9c6fc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.620160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.620778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.621053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.621321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.621990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.623898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.674983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 947/1886] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:42 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:42.676647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.693926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9837-7bc0-4d46-a9b7-3204949c27aa None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.794594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.795281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.795641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.795976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.796678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.798777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.824680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 940/1887] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:42 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:42.828761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.840153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b407-6341-4010-9b2a-ee406c33ebbb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.845088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.845892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.846260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.846623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.847299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.848936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.861467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 948/1888] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:42.866139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.877924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34890-a674-4454-91a7-2ecfe76775b5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.883282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.883905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.884188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.884460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.885116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.886799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec0940-9631-4b72-ae64-720d435b7804 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.898718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 941/1889] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:42.902845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:42.915050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d6c67-2bbc-45aa-9399-103a4c738dd2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:42.920369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:42.921158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.921511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.921896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] PATH_INFO: `/v3/projects/7b28bcdb41114677af2bac0c7b448d51/users/7216ec524c874dcea4e5b66785d06079/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:42.944165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b28bcdb41114677af2bac0c7b448d51, user_id=7216ec524c874dcea4e5b66785d06079, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:42.947491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:42.960462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:42.968361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 949/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:42 2022] PUT /identity/v3/projects/7b28bcdb41114677af2bac0c7b448d51/users/7216ec524c874dcea4e5b66785d06079/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:42.983963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:42.984345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:42.984710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.016219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] MFA Rules not processed for user `7216ec524c874dcea4e5b66785d06079`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:43.017441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:43.021674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:43.093328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 942/1891] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:43.097357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.164104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaceec-4f62-4d1a-8005-f041c3601334 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.168787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.169410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.169711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.169979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.170624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.172529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.202895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 950/1892] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:43.207146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.219518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671879b-b910-432a-982f-55c06d6f7513 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.287631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.288195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.288472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.288799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.289407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.291704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.336318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 943/1893] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:43 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:51:43.340260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.361139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee2f03-6ce2-4190-aa45-f24289dfffe1 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.367604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.368675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.369286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.370163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.371077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.373720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.389398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 951/1894] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:43.393472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.404683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9aae4-697a-4566-9b50-a7637fc25247 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.410072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.410990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.411419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.411875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] PATH_INFO: `/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc/users/f8b7819132c54177985567acf250a615/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.433578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1ba5e495234859abb67f3e4d3f19fc, user_id=f8b7819132c54177985567acf250a615, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.436711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.472122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 944/1895] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:43 2022] PUT /identity/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc/users/f8b7819132c54177985567acf250a615/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:43.476538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.488342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8648-2401-4d22-b819-4dabfabd8f32 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.571763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.572484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.572832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.573171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.574088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.575837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bce7b-7832-426e-bce2-7de818d12a0b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.588016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 952/1896] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:43.593853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.614154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffccf4d-c1d7-4bd2-b798-ef877834f843 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.622344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.623362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.623621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.626094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] PATH_INFO: `/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc/users/f8b7819132c54177985567acf250a615/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.629052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1ba5e495234859abb67f3e4d3f19fc, user_id=f8b7819132c54177985567acf250a615, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.632802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.661034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 945/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:43 2022] PUT /identity/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc/users/f8b7819132c54177985567acf250a615/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:51:43.680597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.681183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.681183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.724797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] MFA Rules not processed for user `f8b7819132c54177985567acf250a615`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:43.726086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:43.795618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:43.824837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 953/1898] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:43.832753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:43.835562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:43.927532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1eac-a6b8-407a-aacc-070ed0f7b422 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:43.932972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:43.933565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:43.933904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:43.934156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:43.945903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:43.947355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:43.952872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:43.959708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 946/1899] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:44.671386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:44.672875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:44.691058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b2601-cfff-43b5-ab67-94ee72bcb5d8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:44.696374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:44.697208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:44.697595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:44.698011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:44.771800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:44.779936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:44.785692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 954/1900] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:51.233806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:51.243883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58d64-79cc-4a38-8ef4-6fd49f12afb3 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:51.253207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:51.254508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:51.255279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:51.256139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:51.337756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:51.339236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967769-bce4-4cf2-b74a-5dafff1e5a63 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:51.348627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 947/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:53.848186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:53.918554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb4e14-3ab1-4fcd-a0e8-84efd06dac01 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:53.923662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:53.924310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:53.924652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:53.924929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] PATH_INFO: `/v3/users/9c8ccfedb13349b68490227921cd2c8b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:53.926211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8ccfedb13349b68490227921cd2c8b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:53.928686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:53.983330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8ccfedb13349b68490227921cd2c8b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:53.991465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8ccfedb13349b68490227921cd2c8b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.004593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8ccfedb13349b68490227921cd2c8b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.005325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8ccfedb13349b68490227921cd2c8b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.013683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8ccfedb13349b68490227921cd2c8b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.020356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 955/1902] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:53 2022] DELETE /identity/v3/users/9c8ccfedb13349b68490227921cd2c8b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:54.025060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:54.025552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:54.025672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:54.046936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:54.048164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:54.148267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 948/1903] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:54.160274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:54.163232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:54.196618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:54.263935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467778a-28d9-401f-b427-b5a50fbde571 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:54.269098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:54.269747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:54.270263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:54.270548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:54.281636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:54.283111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:54.293670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 956/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:54.564294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:54.565800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:54.578985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f010981-0988-46ea-a3fb-9ca15d0e9d35 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:54.584387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:54.584904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:54.585228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:54.585445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] PATH_INFO: `/v3/projects/80e764accca34094b0b320af8515451e` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:54.586930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e764accca34094b0b320af8515451e)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:54.588861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:54.636957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e764accca34094b0b320af8515451e', 'request_id': 'req-cf33460c-d46e-4595-9397-468221c3a03d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.645156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e764accca34094b0b320af8515451e', 'request_id': 'req-cf33460c-d46e-4595-9397-468221c3a03d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.652371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e764accca34094b0b320af8515451e', 'request_id': 'req-cf33460c-d46e-4595-9397-468221c3a03d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.664861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] The token cache is being invalidate because project 80e764accca34094b0b320af8515451e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:54.665151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.665899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.667484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 949/1905] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:54 2022] DELETE /identity/v3/projects/80e764accca34094b0b320af8515451e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:54.671658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:54.681786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0258891-36d1-41f7-8841-bd72ee7ffee7 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:54.739437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:54.741377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:54.741377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:54.741377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] PATH_INFO: `/v3/users/7216ec524c874dcea4e5b66785d06079` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:54.742498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7216ec524c874dcea4e5b66785d06079)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:54.744436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:54.792034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7216ec524c874dcea4e5b66785d06079'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.799459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7216ec524c874dcea4e5b66785d06079'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.814884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7216ec524c874dcea4e5b66785d06079'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.815644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7216ec524c874dcea4e5b66785d06079'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.824595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7216ec524c874dcea4e5b66785d06079'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:54.831305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 957/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:54 2022] DELETE /identity/v3/users/7216ec524c874dcea4e5b66785d06079 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:55.105164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:55.106685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:55.121048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33460c-d46e-4595-9397-468221c3a03d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:55.143122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:55.186046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:55.186598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:55.186878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:55.187153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] PATH_INFO: `/v3/projects/7b28bcdb41114677af2bac0c7b448d51` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:55.188289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b28bcdb41114677af2bac0c7b448d51)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:55.190271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:55.241913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b28bcdb41114677af2bac0c7b448d51', 'request_id': 'req-8431ba28-92aa-432e-a495-b821c5382b8e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.250545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b28bcdb41114677af2bac0c7b448d51', 'request_id': 'req-8431ba28-92aa-432e-a495-b821c5382b8e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.256696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b28bcdb41114677af2bac0c7b448d51', 'request_id': 'req-8431ba28-92aa-432e-a495-b821c5382b8e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.268783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] The token cache is being invalidate because project 7b28bcdb41114677af2bac0c7b448d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:55.269098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.269798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.271434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 950/1907] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:55 2022] DELETE /identity/v3/projects/7b28bcdb41114677af2bac0c7b448d51 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:55.275649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:55.277100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:55.290873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa34fc4-c40f-4ae1-b3cf-d648080bfde6 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:55.334203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:55.361375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:55.362051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:55.362330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:55.362595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] PATH_INFO: `/v3/users/f8b7819132c54177985567acf250a615` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:55.363747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b7819132c54177985567acf250a615)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:55.366897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:55.419039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b7819132c54177985567acf250a615'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.427032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b7819132c54177985567acf250a615'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.441689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b7819132c54177985567acf250a615'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.442318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b7819132c54177985567acf250a615'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.450297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2637deae-a843-40a6-8776-12cc012c5775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b7819132c54177985567acf250a615'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.456432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 958/1908] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:51:55 2022] DELETE /identity/v3/users/f8b7819132c54177985567acf250a615 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:55.759658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:55.761109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:55.774487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431ba28-92aa-432e-a495-b821c5382b8e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:55.836424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:55.836424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:55.836424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:55.836834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] PATH_INFO: `/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:55.837824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1ba5e495234859abb67f3e4d3f19fc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:55.839740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:55.888280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1ba5e495234859abb67f3e4d3f19fc', 'request_id': 'req-a771b2dc-121c-45c0-9079-bd395be7192d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.892102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:55.895649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1ba5e495234859abb67f3e4d3f19fc', 'request_id': 'req-a771b2dc-121c-45c0-9079-bd395be7192d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.903514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1ba5e495234859abb67f3e4d3f19fc', 'request_id': 'req-a771b2dc-121c-45c0-9079-bd395be7192d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.915146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] The token cache is being invalidate because project 9a1ba5e495234859abb67f3e4d3f19fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:51:55.915429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.916067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a771b2dc-121c-45c0-9079-bd395be7192d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:51:55.917549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 951/1909] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:51:55 2022] DELETE /identity/v3/projects/9a1ba5e495234859abb67f3e4d3f19fc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:55.924266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:55.924549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:55.924819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:55.946274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:55.947678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:56.007771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 959/1910] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:56.016558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.016846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.017150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.037628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:56.038871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:56.046859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 952/1911] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:56.051210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.061536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c84877-053c-4a12-af3d-dc8bdfb66fc4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.066424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.067034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.067316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.067588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.068290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:51:56.068576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.070222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.082651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 960/1912] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:51:56.087156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.096440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a896b-2b63-41b6-98e8-24aff2b5a6af None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.102511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.103127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.103412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.103687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.104324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.106298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.142833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 953/1913] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:56.147436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.157907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f64e-58e2-4bb7-a4a6-c8257daeb549 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.203928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:56.223714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.224964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.225252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.225524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.226279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.228248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.263167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 961/1914] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:56.264396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.276199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeae9f-3686-45d9-94b5-474c783001df admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.281276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.281922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.282206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.282482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.283134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.284749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.296095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 954/1915] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:56.301795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.312779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983be7-4cfc-401c-9965-234c1ad967fb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.317576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.318952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.319382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.319659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.320338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.322725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a244b86-263a-482e-9b6e-eacc57bf2e7d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.343894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 962/1916] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:56.348267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.360639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387f4b3-d3bf-49f4-b226-ac8fb50cce65 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.368062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.368737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.369012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.369280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] PATH_INFO: `/v3/projects/75af996e0f0f487e8084b108b0f5a219/users/0bd81901b1f741238bbbe0bf9f042e75/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.391598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75af996e0f0f487e8084b108b0f5a219, user_id=0bd81901b1f741238bbbe0bf9f042e75, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.394358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.409883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 955/1917] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:56 2022] PUT /identity/v3/projects/75af996e0f0f487e8084b108b0f5a219/users/0bd81901b1f741238bbbe0bf9f042e75/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:56.421951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.422309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.422651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.452540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] MFA Rules not processed for user `0bd81901b1f741238bbbe0bf9f042e75`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:56.453889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:56.524240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 963/1918] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:51:56.528595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.619523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880ee21-f261-40c9-86e3-9d92740defaf admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.624623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.625054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.625327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.625591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.626610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.628399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.662145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 956/1919] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:51:56.666941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.682837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dd0cd-5937-4fb8-8264-45235a711e18 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.746893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.747438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.747745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.748084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.748793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.750793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.777682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 964/1920] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:51:56 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:51:56.781544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.794143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f452-b22d-4c96-9d21-b06806e8832c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.799396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.800008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.800288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.800561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.801209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.802870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.817602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 957/1921] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:51:56.821975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.832360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a4c8-a578-48c3-9818-d2d55182a332 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.838021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.838857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.839238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.839407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] PATH_INFO: `/v3/projects/1993c3ed5263404b9f424b009938eae1/users/0a1016d4321445d199abaf3c4b384801/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.854952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1993c3ed5263404b9f424b009938eae1, user_id=0a1016d4321445d199abaf3c4b384801, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.857630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.871287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 965/1922] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:56 2022] PUT /identity/v3/projects/1993c3ed5263404b9f424b009938eae1/users/0a1016d4321445d199abaf3c4b384801/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:56.875287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.885756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a9889-4f81-42bb-828f-d093cb733e68 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.934778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:56.946434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.947549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.948071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.948541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.949195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.950846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d6822-9592-4295-a165-71d0cb326266 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.961433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 958/1923] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:51:56.966668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:56.976917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79cbc3-c2f4-46d3-b8fa-d85a43e5f88d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:56.981452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:56.982143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:56.982418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:56.982687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] PATH_INFO: `/v3/projects/1993c3ed5263404b9f424b009938eae1/users/0a1016d4321445d199abaf3c4b384801/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:56.984580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1993c3ed5263404b9f424b009938eae1, user_id=0a1016d4321445d199abaf3c4b384801, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:56.987241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:56.998052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 966/1924] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:51:56 2022] PUT /identity/v3/projects/1993c3ed5263404b9f424b009938eae1/users/0a1016d4321445d199abaf3c4b384801/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:51:57.010131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:57.010417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:57.010701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:57.039872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] MFA Rules not processed for user `0a1016d4321445d199abaf3c4b384801`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:51:57.041085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:57.112027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 959/1925] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:51:57 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:51:57.124703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:57.126322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:51:57.132999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:51:57.193773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c515c8-bda2-4daf-847f-a8adab4dc62c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:57.200285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:57.201939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:57.202272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:57.202705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:57.215158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:57.216596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:57.226485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 967/1926] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:57 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:51:57.306322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:51:57.316045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fb1e7-1547-4bbf-b1ea-bd9cf93ad0d0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:51:57.319795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:51:57.320400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:51:57.320674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:51:57.320944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:51:57.378551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:51:57.380035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a55360-68b6-4ca5-a35a-54df6ea5c1dc service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:51:57.389613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 960/1927] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:51:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:02.631826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:02.695935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c33f-c413-4396-b181-0b55103348df service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:02.700674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:02.701307 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:02.701581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:02.701877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] PATH_INFO: `/v3/users/0bd81901b1f741238bbbe0bf9f042e75` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:02.702993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd81901b1f741238bbbe0bf9f042e75)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:02.704986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:02.752272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd81901b1f741238bbbe0bf9f042e75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:02.760079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd81901b1f741238bbbe0bf9f042e75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:02.772377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd81901b1f741238bbbe0bf9f042e75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:02.773022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd81901b1f741238bbbe0bf9f042e75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:02.781501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd81901b1f741238bbbe0bf9f042e75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:02.787411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 968/1928] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:02 2022] DELETE /identity/v3/users/0bd81901b1f741238bbbe0bf9f042e75 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:02.792540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:02.793123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:02.793123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:02.812259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:02.813471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:02.874253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 961/1929] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:02.884949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:02.886461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:02.918142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:02.955669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994dd67-cda8-4ff7-87a1-3744d8280371 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:02.966425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:02.967497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:02.968100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:02.968787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:02.980547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:02.982043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:02.992994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 969/1930] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:52:03.278891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:03.280924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:03.300196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43137ea-cd64-45af-bbbb-34cef1b7c634 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:03.305421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:03.306082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:03.306406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:03.306605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] PATH_INFO: `/v3/projects/75af996e0f0f487e8084b108b0f5a219` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:03.308073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75af996e0f0f487e8084b108b0f5a219)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:03.310088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:03.318786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:03.361390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75af996e0f0f487e8084b108b0f5a219', 'request_id': 'req-37c9a0ce-6cfa-433a-9a63-ed054551b30c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.370370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75af996e0f0f487e8084b108b0f5a219', 'request_id': 'req-37c9a0ce-6cfa-433a-9a63-ed054551b30c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.376244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75af996e0f0f487e8084b108b0f5a219', 'request_id': 'req-37c9a0ce-6cfa-433a-9a63-ed054551b30c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.387225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] The token cache is being invalidate because project 75af996e0f0f487e8084b108b0f5a219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:03.387530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.388155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.389681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 962/1931] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:03 2022] DELETE /identity/v3/projects/75af996e0f0f487e8084b108b0f5a219 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:03.394177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:03.404093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83f159-a390-4216-ba58-1ab5b2f85d44 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:03.461899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:03.462422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:03.462703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:03.462975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] PATH_INFO: `/v3/users/0a1016d4321445d199abaf3c4b384801` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:03.464116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1016d4321445d199abaf3c4b384801)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:03.466083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:03.514359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1016d4321445d199abaf3c4b384801'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.522113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1016d4321445d199abaf3c4b384801'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.534589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1016d4321445d199abaf3c4b384801'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.535224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1016d4321445d199abaf3c4b384801'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.543028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9667e3fb-22af-4c8b-a89d-56ec97393b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1016d4321445d199abaf3c4b384801'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:03.549009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 970/1932] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:03 2022] DELETE /identity/v3/users/0a1016d4321445d199abaf3c4b384801 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:03.863155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:03.864674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:03.879682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a0ce-6cfa-433a-9a63-ed054551b30c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:03.940950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:03.941605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:03.941917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:03.942185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] PATH_INFO: `/v3/projects/1993c3ed5263404b9f424b009938eae1` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:03.943327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1993c3ed5263404b9f424b009938eae1)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:03.945300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:03.997737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1993c3ed5263404b9f424b009938eae1', 'request_id': 'req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:04.005475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1993c3ed5263404b9f424b009938eae1', 'request_id': 'req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:04.012502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1993c3ed5263404b9f424b009938eae1', 'request_id': 'req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:04.027621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] The token cache is being invalidate because project 1993c3ed5263404b9f424b009938eae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:04.027916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:04.028588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9a7af-54ef-4b6b-bd78-b8cbe6578bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:04.030275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 963/1933] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:03 2022] DELETE /identity/v3/projects/1993c3ed5263404b9f424b009938eae1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:04.038730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.039889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.040213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.066240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:04.066240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:04.126555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:04.153883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 971/1934] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:04.163703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.164169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.164482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.185017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:04.186257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:04.194776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 964/1935] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:04.199919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.212186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6c8a-090c-4e9b-a141-04016aefc495 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.217040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.217712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.218028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.218330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.219015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:04.219322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.220947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.237219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 972/1936] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:04.238055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.247867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48009b68-facb-43e2-a4ca-b394b8dbacdc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.254697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.255472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.255848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.256288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.257357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.263304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.288544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:04.301677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 965/1937] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:04.302991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.314008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f784b0-45ce-4342-a149-b11d9d97423e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.379046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.379667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.380064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.380352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.380991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.383521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.409536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 973/1938] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:04.411152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.422824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3553607-e47d-4453-a57a-ed3b682726ec admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.427647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.428246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.428522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.428791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.429972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.432310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.443245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 966/1939] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:04.448998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.460484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a65da-052d-4f47-84cb-ad2ceeec4747 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.466065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.466719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.467019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.467310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.467961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.469567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04e3ef-ec91-4b1c-82de-8c11ff94c80c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.480729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 974/1940] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:04.485655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.497074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa523-5b67-4cdb-8155-67c04816e16b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.502483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.503163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.503444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.503721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] PATH_INFO: `/v3/projects/9d960dd95a28473ab973bf0b24b3237b/users/78e6630aceba4311838f8a6149ec0422/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.520494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d960dd95a28473ab973bf0b24b3237b, user_id=78e6630aceba4311838f8a6149ec0422, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.523205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.536507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 967/1941] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:04 2022] PUT /identity/v3/projects/9d960dd95a28473ab973bf0b24b3237b/users/78e6630aceba4311838f8a6149ec0422/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:04.548838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.549126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.549427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.578918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] MFA Rules not processed for user `78e6630aceba4311838f8a6149ec0422`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:04.580163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:04.646094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 975/1942] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:04.650743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.712643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee51ad1-c7c4-4519-966b-c856f98885de admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.717480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.718114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.718393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.718681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.719328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.721364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.760847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 968/1943] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:04.765262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.777304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106fef0-aeff-4a8d-bb8e-fb761e42a2b1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.829012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:04.850409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.851192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.852086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.852415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.853235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.855636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.880063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 976/1944] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:04 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:04.884789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.897125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4e87-cfa2-4848-b3dc-4cf1776d89aa admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.902679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.903298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.903574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.903846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.904492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.906175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.918694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 969/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:04.923555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:04.935316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8e879-9882-4cdf-9e62-b0f5c2aa0c15 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:04.940224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:04.941023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:04.941309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:04.941656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] PATH_INFO: `/v3/projects/423a81b533ad4d36bd755dbc3107ea7e/users/b4db4b076fc645ee86c60530be81476e/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:04.967368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423a81b533ad4d36bd755dbc3107ea7e, user_id=b4db4b076fc645ee86c60530be81476e, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:04.970445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:04.986308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 977/1946] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:04 2022] PUT /identity/v3/projects/423a81b533ad4d36bd755dbc3107ea7e/users/b4db4b076fc645ee86c60530be81476e/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:04.990757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:05.001075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264760e-4fb4-448a-8ac1-5c1b7ce9cd3c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:05.065828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:05.066440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.066719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.067004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.067635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:05.069252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153d299-5772-4ce0-95a4-e574a39f98ee admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:05.083736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 970/1947] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:05.085472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:05.098088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199869ee-7655-465a-909e-247a5cc8f390 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:05.103744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:05.104557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.104786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.106994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] PATH_INFO: `/v3/projects/423a81b533ad4d36bd755dbc3107ea7e/users/b4db4b076fc645ee86c60530be81476e/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.106994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423a81b533ad4d36bd755dbc3107ea7e, user_id=b4db4b076fc645ee86c60530be81476e, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:05.109945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:05.123236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 978/1948] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:05 2022] PUT /identity/v3/projects/423a81b533ad4d36bd755dbc3107ea7e/users/b4db4b076fc645ee86c60530be81476e/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:05.133978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.134282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.134563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.164133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] MFA Rules not processed for user `b4db4b076fc645ee86c60530be81476e`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:05.165362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:05.185703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:05.234599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 971/1949] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:05.246138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:05.247864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:05.321388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71369-aa04-4e21-aa5c-34e9db2725c0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:05.325173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:05.325801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.326085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.326359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.398456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:05.399920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183c35c-2959-437e-8024-7fcb9c84b40e service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:05.409417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 979/1950] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:05.443132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:05.444673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:05.516041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec2370-eb2a-4026-b4c2-6e1f21623685 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:05.521120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:05.521797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.522176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.522351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] PATH_INFO: `/v3/users/5e6049ea6c874b8e8623a9229ab46c3c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.526578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6049ea6c874b8e8623a9229ab46c3c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:05.529195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:05.580995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6049ea6c874b8e8623a9229ab46c3c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:05.581664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6049ea6c874b8e8623a9229ab46c3c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:05.600007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6049ea6c874b8e8623a9229ab46c3c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:05.609175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6049ea6c874b8e8623a9229ab46c3c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:05.609926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6049ea6c874b8e8623a9229ab46c3c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:05.625542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 972/1951] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:05 2022] DELETE /identity/v3/users/5e6049ea6c874b8e8623a9229ab46c3c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:05.630514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.630899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.631355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.651700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:05.653045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:05.702852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:05.722548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 980/1952] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:05.733790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:05.735347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:05.805539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:05.824161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6721-385b-40fc-9344-6ce673b1cc53 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:05.827936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:05.828581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:05.828858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:05.829132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:05.839126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:05.840579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb22db-44ce-491f-b952-7fac9a4f5b9a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:05.850732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 973/1953] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:06.193159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.196527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:06.209793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e111a41-3a4d-4642-ae5b-889122744ece None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.215642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.216303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.216582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.216853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] PATH_INFO: `/v3/projects/eb53ec5b53bf4cd385ea1088a392e2d5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.218145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb53ec5b53bf4cd385ea1088a392e2d5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.220078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.283499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb53ec5b53bf4cd385ea1088a392e2d5', 'request_id': 'req-e786d8ca-c9b3-44dc-9464-58121f949644'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:06.289765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb53ec5b53bf4cd385ea1088a392e2d5', 'request_id': 'req-e786d8ca-c9b3-44dc-9464-58121f949644'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:06.297309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb53ec5b53bf4cd385ea1088a392e2d5', 'request_id': 'req-e786d8ca-c9b3-44dc-9464-58121f949644'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:06.308744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] The token cache is being invalidate because project eb53ec5b53bf4cd385ea1088a392e2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:06.309030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:06.309703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e786d8ca-c9b3-44dc-9464-58121f949644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:06.311203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 981/1954] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:06 2022] DELETE /identity/v3/projects/eb53ec5b53bf4cd385ea1088a392e2d5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:06.316129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.316604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.316725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.338090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:06.339507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:06.404424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 974/1955] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:06.412308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.412682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.412958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.433071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:06.434662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:06.443435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 982/1956] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:06.448825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.459635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a3f45-c5ca-4c6e-89f9-633652731e5e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.465122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.465768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.466049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.466321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.467034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:06.467339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.469084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.482203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 975/1957] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:06.486574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.496064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ad2a3-8569-407f-84ae-b77a3bfb43a5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.501003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.501607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.502008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.502278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.502949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.504851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.533157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 983/1958] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:06 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:06.537389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.548766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e244734-8ec2-45a8-8d71-ea056179a72c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.608873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.609481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.609785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.610055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.610691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.612609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.636116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 976/1959] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:06 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:06.641498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.653204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ffadc-3a34-4bd8-8ef7-c7f8a4f33122 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.658635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.659239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.659509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.659775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.660416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.662046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.672033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 984/1960] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:06.675977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.687026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6a71-9964-48c4-ac41-56bb519e80f5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.691792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.692386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.692661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.692932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.693569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.695935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450570c-8443-4080-b210-6ba9a629b09a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.706780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 977/1961] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:06.710992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.721318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e783b-2e8e-4a1b-b838-99e4432a2f70 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.726066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.726725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.727003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.727280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] PATH_INFO: `/v3/projects/bb69363112bd4ac9af8e6a8cf4213d95/users/3cf9ce52e7de483c9749daa4d88b0358/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.742704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb69363112bd4ac9af8e6a8cf4213d95, user_id=3cf9ce52e7de483c9749daa4d88b0358, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:06.745581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:06.759770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 985/1962] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:06 2022] PUT /identity/v3/projects/bb69363112bd4ac9af8e6a8cf4213d95/users/3cf9ce52e7de483c9749daa4d88b0358/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:06.770251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.770537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.770807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:06.802186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] MFA Rules not processed for user `3cf9ce52e7de483c9749daa4d88b0358`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:06.803600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:06.845972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:06.884193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 978/1963] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:06.921220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:06.923482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:06.988824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fc23f-0dcd-42f8-bb46-f4dc65b8d089 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:06.993463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:06.994128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:06.994406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:06.994691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.004079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:07.005549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701ed8f-dad1-4826-909c-37acf3f2b61f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:07.016446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 986/1964] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:07.031926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:07.099132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf202e1-8d55-4307-9885-c9c9cb8b112b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:07.104854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:07.105486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.105795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.106070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] PATH_INFO: `/v3/users/3cf9ce52e7de483c9749daa4d88b0358` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.107212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf9ce52e7de483c9749daa4d88b0358)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:07.109137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:07.159105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf9ce52e7de483c9749daa4d88b0358'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.159766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf9ce52e7de483c9749daa4d88b0358'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.172472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf9ce52e7de483c9749daa4d88b0358'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.181564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf9ce52e7de483c9749daa4d88b0358'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.182317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf9ce52e7de483c9749daa4d88b0358'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.199201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 979/1965] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:07 2022] DELETE /identity/v3/users/3cf9ce52e7de483c9749daa4d88b0358 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:07.204037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.204457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.204703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.227399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:07.228660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:07.234217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:07.300723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 987/1966] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:07.309085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:07.310726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:07.369989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b6e84-a0f7-492a-af57-5ee3202b7f47 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:07.373739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:07.374367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.374737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.374907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.385835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:07.387284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ee33d-47c4-4fec-becb-dcd0e5897dea service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:07.391739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:07.400943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 980/1967] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:07.699860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:07.701369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:07.720878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3d7b0-a1c7-4834-8630-9730b182f0ab None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:07.727008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:07.728468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.728749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.729332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] PATH_INFO: `/v3/projects/bb69363112bd4ac9af8e6a8cf4213d95` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.730761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb69363112bd4ac9af8e6a8cf4213d95)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:07.732814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:07.797471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb69363112bd4ac9af8e6a8cf4213d95', 'request_id': 'req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.803005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb69363112bd4ac9af8e6a8cf4213d95', 'request_id': 'req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.814437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb69363112bd4ac9af8e6a8cf4213d95', 'request_id': 'req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.828865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] The token cache is being invalidate because project bb69363112bd4ac9af8e6a8cf4213d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:07.829156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.829841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-68d3db36-0c15-4c92-9ffc-15c2cf55e5d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:07.831352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 988/1968] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:07 2022] DELETE /identity/v3/projects/bb69363112bd4ac9af8e6a8cf4213d95 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:07.837454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.837775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.838052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.859716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:07.860920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:07.926996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 981/1969] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:07.935913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.936280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.936548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.956927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:07.958160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:07.966968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 989/1970] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:07.971209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:07.981177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10e872-4380-4492-ac8a-4eca8901a265 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:07.987375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:07.987994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:07.988273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:07.988607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:07.989272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:07.989578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:07.991259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.003558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 982/1971] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:08.008520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.018305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df3ad-75f2-44e2-a58c-f019e62dc1b4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.024408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.025003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.025314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.025538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.026217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.028139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.056673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 990/1972] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:08 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:08.060981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.071946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9472724-cae8-4f4c-b38c-9e6884b71af8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.139880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.140761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.141037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.141307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.141986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.144376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.168560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 983/1973] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:08 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:08.172987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.187863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c848e-c819-4624-9458-017af1d7b15d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.193422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.194094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.194372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.194660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.196934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.198926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.213167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 991/1974] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:08.215109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.227756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8ab4b-1a5d-4305-8dc1-2630b07b87ab admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.233181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.234148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.234436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.235101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.235350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.238284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3093d-c972-42bf-a422-ba37d8eaa141 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.264178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.264338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 984/1975] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:08.271627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9df0-7b6a-442a-8625-4400d9387b7f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.278195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.278912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.279262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.279693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] PATH_INFO: `/v3/projects/0a6ca362b183420f854c9f0bdee5d429/users/f11aea69eecb49bf98b4a5a551c51346/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.298646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6ca362b183420f854c9f0bdee5d429, user_id=f11aea69eecb49bf98b4a5a551c51346, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.301652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.317785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 992/1976] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:08 2022] PUT /identity/v3/projects/0a6ca362b183420f854c9f0bdee5d429/users/f11aea69eecb49bf98b4a5a551c51346/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:08.328142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.328429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.328699 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.358586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] MFA Rules not processed for user `f11aea69eecb49bf98b4a5a551c51346`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:08.359790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:08.414460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:08.426166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 985/1977] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:08.436968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:08.438642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:08.475425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:08.503896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd260-6329-4903-97a2-59305feffee7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:08.507588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:08.508203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:08.508479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:08.508753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:08.518005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:08.519451 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283c00f-d47a-4e75-a04f-535028074cd8 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:08.529262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 993/1978] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:12.680284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:12.752590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992e7a-f36e-46c1-bc30-317b03d7e73b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:12.757405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:12.758221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:12.758559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:12.758843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] PATH_INFO: `/v3/users/f11aea69eecb49bf98b4a5a551c51346` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:12.760092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11aea69eecb49bf98b4a5a551c51346)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:12.762086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:12.814361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11aea69eecb49bf98b4a5a551c51346'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:12.815150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11aea69eecb49bf98b4a5a551c51346'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:12.827556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11aea69eecb49bf98b4a5a551c51346'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:12.838126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11aea69eecb49bf98b4a5a551c51346'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:12.838778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11aea69eecb49bf98b4a5a551c51346'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:12.852753 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 986/1979] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:12 2022] DELETE /identity/v3/users/f11aea69eecb49bf98b4a5a551c51346 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:12.857443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:12.857931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:12.858044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:12.877733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:12.878967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:12.945585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 994/1980] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:12.952563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:12.954066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.013292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5afd6-5c29-4017-bce1-d11f958331cf admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.017159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.017811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.018209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.018435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.029033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.031659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.044485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 987/1981] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:13.059858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.061352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.130706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdffa6-49a1-4f7c-9ab3-0ccf3e014591 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.134598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.135282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.135559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.135825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.137693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.210953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.212449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476c801-91b2-4cba-b856-0f62a485929e service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.222911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 995/1982] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:13.384233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.386357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.400729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74300c-bbc9-4adc-b7de-f60af42b4205 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.406107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.406658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.406976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.407204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] PATH_INFO: `/v3/projects/0a6ca362b183420f854c9f0bdee5d429` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.408345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6ca362b183420f854c9f0bdee5d429)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.410371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.423108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:13.470109 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6ca362b183420f854c9f0bdee5d429', 'request_id': 'req-979586eb-f6ff-4e0b-be1c-8b1a5682993f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:13.480431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6ca362b183420f854c9f0bdee5d429', 'request_id': 'req-979586eb-f6ff-4e0b-be1c-8b1a5682993f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:13.487808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6ca362b183420f854c9f0bdee5d429', 'request_id': 'req-979586eb-f6ff-4e0b-be1c-8b1a5682993f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:13.502076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] The token cache is being invalidate because project 0a6ca362b183420f854c9f0bdee5d429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:13.502449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:13.503024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-979586eb-f6ff-4e0b-be1c-8b1a5682993f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:13.505141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 988/1983] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:13 2022] DELETE /identity/v3/projects/0a6ca362b183420f854c9f0bdee5d429 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:13.510829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.511216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.511583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.536695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:13.538162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.569035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:13.643892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 996/1984] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:13.653417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.654036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.654551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.680696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:13.682124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:13.690666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 989/1985] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:13.695292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.706923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5f2ac-5753-4a12-a894-6ae868838364 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.712249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.713020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.713306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.713729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.714389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:13.714697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.716296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.731323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 997/1986] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:13.734688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.748989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed52561-b208-4a9d-a4a8-d3daf4df96b3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.754196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.754744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.755075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.755278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.755913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.757976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.790960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 990/1987] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:13 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:13.797882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.805911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de229-a90a-4816-aa99-b23d687048d7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.865771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.866435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.866837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.866953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.867561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.869494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.893875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 998/1988] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:13 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:13.898640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.914389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0476-eea5-48f1-ae46-c13eb51ee7ff admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.921204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.921907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.922198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.922472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.923124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.924751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.939228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 991/1989] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:13.940435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.951557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352525-d9ef-429f-8384-bf2583873f6b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.957549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.958222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.958501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.958770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:13.959414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:13.961824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6ee96-a0f2-4cf1-abb8-09accb4e9330 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:13.973906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 999/1990] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:13.978670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:13.989812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f88106-8b1b-46c4-8d3e-a227633a3f29 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:13.994554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:13.995269 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:13.995783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:13.995783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] PATH_INFO: `/v3/projects/a299dd53e9dd41f6ae5e30fadb0f1d50/users/7b3df765a48d4a23be6d989a77cfd0af/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.012324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a299dd53e9dd41f6ae5e30fadb0f1d50, user_id=7b3df765a48d4a23be6d989a77cfd0af, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.015103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.032173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 992/1991] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:13 2022] PUT /identity/v3/projects/a299dd53e9dd41f6ae5e30fadb0f1d50/users/7b3df765a48d4a23be6d989a77cfd0af/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:14.053041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.055036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.056808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.109850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] MFA Rules not processed for user `7b3df765a48d4a23be6d989a77cfd0af`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:14.109850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:14.176275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1000/1992] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:14.181606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.247209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84535d5-a75a-42b1-94a8-d2261dad614d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.251924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.252544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.252824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.253189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.253762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.255674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.283903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 993/1993] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:14.288050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.299578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c183d8-ce24-40a6-bb2c-deadb0a0da26 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.316556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:14.360851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.361461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.361768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.362036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.362681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.364578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.389287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1001/1994] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:14.391666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.402287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eeacc-a521-46ed-bb74-db61a63b1a9e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.407177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.407777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.408054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.408320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.408993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.410597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.422157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 994/1995] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:14.426218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.437519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89ff60-d86f-4ca9-9fcc-6f9ccf37eb74 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.442402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.443016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.443296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.443567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.444214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.446734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9e4f-f83f-4fbc-ae2d-95ddb59f8e40 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.457899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1002/1996] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:14.462196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.472375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a574a-8fde-43c3-9a79-3aecda4a6f6c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.477912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.478588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.478867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.479143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] PATH_INFO: `/v3/projects/fe9a05b18dff4485be0e1c9433acc4b9/users/b46c34eabbf7495291f28428103ce612/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.483384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:14.499078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9a05b18dff4485be0e1c9433acc4b9, user_id=b46c34eabbf7495291f28428103ce612, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.502218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.517955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 995/1997] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:14 2022] PUT /identity/v3/projects/fe9a05b18dff4485be0e1c9433acc4b9/users/b46c34eabbf7495291f28428103ce612/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:14.528629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.528906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.529201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.559802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] MFA Rules not processed for user `b46c34eabbf7495291f28428103ce612`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:14.561043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:14.673117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1003/1998] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:14.677517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.755454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca299-7842-45b6-8681-7d2f0544c761 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.761474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.762414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.762883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.763550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.764313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.766830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.799915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 996/1999] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:14.805227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.819919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac3b5-945d-4740-bcf3-1fea62a61089 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.908414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.909396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.909683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.909958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.910613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.912619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:14.945688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1004/2000] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:14 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:14.949262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:14.974526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196107-488a-4de6-98d9-c9bb4b598677 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:14.980226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:14.980847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:14.981515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:14.981515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:14.982793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:14.984409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:15.001712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 997/2001] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:15.003802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:15.020045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f7bbf-6283-4703-b80d-dd72576ec0f8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:15.026175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:15.026881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:15.027145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:15.027496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] PATH_INFO: `/v3/projects/dd0be304bd2e445280ba09f69efdaaf3/users/7d87b81ec4d24a9785070e5822316daa/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:15.045747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0be304bd2e445280ba09f69efdaaf3, user_id=7d87b81ec4d24a9785070e5822316daa, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:15.049716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:15.052905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:15.067800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1005/2002] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:15 2022] PUT /identity/v3/projects/dd0be304bd2e445280ba09f69efdaaf3/users/7d87b81ec4d24a9785070e5822316daa/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:15.072964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:15.083475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5fd32-c511-4fe6-87fd-2c0c021bb189 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:15.154610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:15.155257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:15.155589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:15.155806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:15.156451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:15.158115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a79ef5-3833-4d22-8fad-29eaaf8349c8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:15.169939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 998/2003] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:15.174724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:15.189258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56b6-f9a5-4726-9ce6-32e7064bacbd admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:15.196277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:15.196958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:15.197223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:15.197492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] PATH_INFO: `/v3/projects/dd0be304bd2e445280ba09f69efdaaf3/users/7d87b81ec4d24a9785070e5822316daa/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:15.200231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0be304bd2e445280ba09f69efdaaf3, user_id=7d87b81ec4d24a9785070e5822316daa, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:15.203321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:15.222279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1006/2004] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:15 2022] PUT /identity/v3/projects/dd0be304bd2e445280ba09f69efdaaf3/users/7d87b81ec4d24a9785070e5822316daa/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:15.237261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:15.237547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:15.237854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:15.280323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] MFA Rules not processed for user `7d87b81ec4d24a9785070e5822316daa`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:15.282085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:15.345034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:15.387521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 999/2005] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:15.400353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:15.402214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:15.470654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeff830-382e-4e6e-9e1f-023b389559c9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:15.477590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:15.478326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:15.478673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:15.478953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:15.552722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:15.554748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:15.572190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1007/2006] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:17.163003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:17.178402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93329f4a-a5be-4a82-823d-7d392216a346 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:17.184611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:17.185677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:17.186213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:17.186667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:17.280472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:17.282789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:17.295243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1000/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:17.973718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:17.975374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:17.987260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497f2e2-3361-4f0a-b5c4-e360257eb1d2 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:17.994570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:17.995196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:17.995473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:17.995741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:18.006442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:18.009356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1873f89-638b-419c-ace7-39b7baec28db service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:18.028315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1008/2008] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:18.461588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:18.464323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:18.574884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf94acb-b598-459e-a00c-116c84650294 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:18.580184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:18.580911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:18.581258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:18.581604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] PATH_INFO: `/v3/users/78e6630aceba4311838f8a6149ec0422` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:18.583341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e6630aceba4311838f8a6149ec0422)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:18.585921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:18.641589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e6630aceba4311838f8a6149ec0422'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:18.643215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e6630aceba4311838f8a6149ec0422'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:18.659614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e6630aceba4311838f8a6149ec0422'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:18.669174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e6630aceba4311838f8a6149ec0422'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:18.669985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e6630aceba4311838f8a6149ec0422'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:18.678885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:18.685195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1001/2009] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:18 2022] DELETE /identity/v3/users/78e6630aceba4311838f8a6149ec0422 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:18.690065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:18.690350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:18.690628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:18.709938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:18.711780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:18.776891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1009/2010] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:18.786594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:18.789162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:18.857488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcc0ed-0efe-4e0a-acce-bce61d8fbba3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:18.863843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:18.865501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:18.865823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:18.866146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:18.886629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:18.889012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:18.905042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1002/2011] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:19.249471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:19.251110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:19.268511 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202f30-62d0-454e-a402-638de0a8caad None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:19.283219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:19.283856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:19.284130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:19.284523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] PATH_INFO: `/v3/projects/9d960dd95a28473ab973bf0b24b3237b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:19.286072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d960dd95a28473ab973bf0b24b3237b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:19.291811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:19.313556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:19.359013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d960dd95a28473ab973bf0b24b3237b', 'request_id': 'req-c8eb0823-7809-489f-abf9-1285ab41b03b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.382032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d960dd95a28473ab973bf0b24b3237b', 'request_id': 'req-c8eb0823-7809-489f-abf9-1285ab41b03b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.399377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d960dd95a28473ab973bf0b24b3237b', 'request_id': 'req-c8eb0823-7809-489f-abf9-1285ab41b03b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.429435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] The token cache is being invalidate because project 9d960dd95a28473ab973bf0b24b3237b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:19.429981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.431052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.433768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1010/2012] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:19 2022] DELETE /identity/v3/projects/9d960dd95a28473ab973bf0b24b3237b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:19.440395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:19.459453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc1894-65a5-4193-bf69-4d0fb3e84699 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:19.616134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:19.618350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:19.619046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:19.619668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] PATH_INFO: `/v3/users/b4db4b076fc645ee86c60530be81476e` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:19.624693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4db4b076fc645ee86c60530be81476e)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:19.627496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:19.727701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4db4b076fc645ee86c60530be81476e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.727701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4db4b076fc645ee86c60530be81476e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.743069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4db4b076fc645ee86c60530be81476e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.756222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4db4b076fc645ee86c60530be81476e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.758303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6797be52-7a74-4c20-87e1-6342268874a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4db4b076fc645ee86c60530be81476e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:19.793421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1003/2013] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:19 2022] DELETE /identity/v3/users/b4db4b076fc645ee86c60530be81476e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:20.287864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:20.293100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:20.307722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb0823-7809-489f-abf9-1285ab41b03b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:20.380244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:20.380833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.381283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.381633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] PATH_INFO: `/v3/projects/423a81b533ad4d36bd755dbc3107ea7e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.382852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] RBAC: Authorizing `identity:delete_project(project_id=423a81b533ad4d36bd755dbc3107ea7e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:20.384870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:20.450380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423a81b533ad4d36bd755dbc3107ea7e', 'request_id': 'req-d186926f-ea83-48c1-ae34-44a51e4c868a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:20.456471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423a81b533ad4d36bd755dbc3107ea7e', 'request_id': 'req-d186926f-ea83-48c1-ae34-44a51e4c868a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:20.465315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423a81b533ad4d36bd755dbc3107ea7e', 'request_id': 'req-d186926f-ea83-48c1-ae34-44a51e4c868a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:20.479328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] The token cache is being invalidate because project 423a81b533ad4d36bd755dbc3107ea7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:20.479596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:20.480314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d186926f-ea83-48c1-ae34-44a51e4c868a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:20.481940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1011/2014] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:20 2022] DELETE /identity/v3/projects/423a81b533ad4d36bd755dbc3107ea7e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:20.488009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.488301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.488629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.522026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:20.524275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:20.540660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:20.604517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1004/2015] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:20.614094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.614492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.614902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.651227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:20.653060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:20.669029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1012/2016] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:20.674837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:20.691737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fd46d-8244-47cb-b8fe-f2766bd6e08a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:20.700393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:20.701344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.701879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.702364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.703596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:20.704224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:20.707306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bd375-3c93-47c9-9ba7-be9688858104 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:20.739192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1005/2017] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:52:20.741994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:20.742755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114afb-7423-4dff-a59f-b9bc53091c4e None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.743110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114afb-7423-4dff-a59f-b9bc53091c4e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.743450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114afb-7423-4dff-a59f-b9bc53091c4e None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.745101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1006/2018] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:52:20.754299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e24a-2134-46fa-b3ce-128bff173db5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:20.761926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:20.762722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.763003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.763263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.764087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:20.766365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:20.807824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1013/2019] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:20 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:20.812565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:20.812958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:20.813143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:20.813432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:20.836180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:20.870361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:20.872785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:20.983881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd55b-ec1a-4e16-aebc-913ff29d53ce admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.014767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.015752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.016290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.016718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.018468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.021693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.065826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1014/2020] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:20 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:21.066780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.072542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.098463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc81d7-108a-477f-b3bb-9c4571cdcce0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.102413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1007/2021] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:52:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:21.103776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.114995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.115662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.115942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.116215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] PATH_INFO: `/v3/users/8dd2f6cc960242e2b01df2caf0aba2cd` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.120306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd2f6cc960242e2b01df2caf0aba2cd)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.122346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.127209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72db17c-e775-4117-8c4c-ed1db26b6c97 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.136543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.137174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.137456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.137773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.138387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.140019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.182447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1008/2022] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:21.183451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.200955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791d2d7-43e4-4112-afd2-ba3b77f1ac27 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.206754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.207649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.207964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.208440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.209167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.211893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93352-0a33-4d93-87a8-167db7876801 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.226848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1009/2023] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:21.228610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292267dd-9ffd-45b4-8220-5abd6bc89e7f None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.228895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292267dd-9ffd-45b4-8220-5abd6bc89e7f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.229172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292267dd-9ffd-45b4-8220-5abd6bc89e7f None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.231692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1010/2024] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 6 11:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:52:21.232589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292267dd-9ffd-45b4-8220-5abd6bc89e7f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.250037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd2f6cc960242e2b01df2caf0aba2cd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.254198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd2f6cc960242e2b01df2caf0aba2cd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.261338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292267dd-9ffd-45b4-8220-5abd6bc89e7f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.273259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.274186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.274469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.274746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] PATH_INFO: `/v3/projects/474249b02d3140b6bae6ec312cdd142f/users/683b8a5c6d9048fe871fc563c9e45ca9/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.285120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd2f6cc960242e2b01df2caf0aba2cd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.286674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd2f6cc960242e2b01df2caf0aba2cd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.302633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c64b66a3-915b-4c5c-96a4-4c720bceefdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd2f6cc960242e2b01df2caf0aba2cd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.317723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=474249b02d3140b6bae6ec312cdd142f, user_id=683b8a5c6d9048fe871fc563c9e45ca9, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.317723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c9ad5-caa6-46eb-8ac8-85b9b4786a9c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.318250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1015/2025] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:21 2022] DELETE /identity/v3/users/8dd2f6cc960242e2b01df2caf0aba2cd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:21.318250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.318250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.318250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.339835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] MFA Rules not processed for user `8fd64b822c934dada9d93d8179e374ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:21.340961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1011/2026] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:21 2022] PUT /identity/v3/projects/474249b02d3140b6bae6ec312cdd142f/users/683b8a5c6d9048fe871fc563c9e45ca9/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:21.342623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.345832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.346186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.346573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.380698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:21.381959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.457021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1016/2027] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:21.466135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:21.467534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.474574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:21.480175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebfb36-8ea1-483d-9e19-9c1d6ba31196 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.482325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1012/2028] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:21.487348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.488384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.490659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.490868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.494865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.496067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.498277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.511934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.513799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.526950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1017/2029] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:52:21.562130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.563533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.576964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca1511-65c7-47de-87f7-2b139b1332b2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.578678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31cb4c-9c66-42c1-877b-65b1daf2fd31 service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.581161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.581906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.582184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.582451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.583749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.584382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.585393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.585393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] PATH_INFO: `/v3/projects/df65a60541a8492a912cab1a8f243253` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.586507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df65a60541a8492a912cab1a8f243253)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.588483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.597304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.607612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11adbc5-11e3-48c5-95f2-15e904fe0375 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.620296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1013/2030] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:52:21.654689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df65a60541a8492a912cab1a8f243253', 'request_id': 'req-066fda00-e163-4d76-8460-42ece8c7f97d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.668807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df65a60541a8492a912cab1a8f243253', 'request_id': 'req-066fda00-e163-4d76-8460-42ece8c7f97d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.678449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df65a60541a8492a912cab1a8f243253', 'request_id': 'req-066fda00-e163-4d76-8460-42ece8c7f97d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.691203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] The token cache is being invalidate because project df65a60541a8492a912cab1a8f243253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:21.691498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.692229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-066fda00-e163-4d76-8460-42ece8c7f97d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:21.693953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1018/2031] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:21 2022] DELETE /identity/v3/projects/df65a60541a8492a912cab1a8f243253 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:21.698982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.699510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.699510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.721266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:21.722575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.811694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1014/2032] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:21.820555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.820926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.821273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.843240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:21.844514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:21.854359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1019/2033] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:21.858752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.869854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183f8c-0328-4321-b740-a23af05f0d49 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.875045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.875719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.875973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.876245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.877298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:21.877298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.878905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.893173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1015/2034] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:52:21.898224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.907822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4949f-f169-4a70-9b16-43537f282bfe None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:21.912771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:21.914035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:21.914339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:21.914609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:21.915329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:21.917243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:21.952317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1020/2035] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:21.956624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:21.967987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa22a8-ea80-4708-8a09-fabfdef5bb47 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.059677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.060367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.060645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.060911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.061806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.063931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.092852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1016/2036] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:21 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:22.097302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.111296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb81944-591c-423a-a325-64156186a3bd admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.116633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.117446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.117752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.118024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.118687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.120757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.134014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1021/2037] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:22.138067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.148869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af562ce-78cd-47d3-9117-65b1679cd02d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.154001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.154731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.155141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.155436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.156241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.158255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c606b6-d065-4e4e-90e4-fb79d44ebc21 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.182356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1017/2038] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:22.187401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.203263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d9fc-b155-4727-8a7a-4a22b24df655 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.213030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.213753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.214041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.214317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] PATH_INFO: `/v3/projects/8e354abdb1804c8cb360087909d86cb7/users/4224d4be170041b89a8057838d99887b/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.232707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e354abdb1804c8cb360087909d86cb7, user_id=4224d4be170041b89a8057838d99887b, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.235569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.269146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1022/2039] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:22 2022] PUT /identity/v3/projects/8e354abdb1804c8cb360087909d86cb7/users/4224d4be170041b89a8057838d99887b/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:22.281848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.282231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.282594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.297926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:22.319709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] MFA Rules not processed for user `4224d4be170041b89a8057838d99887b`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:22.321217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:22.487875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1018/2040] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:22.492074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.573183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.575401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:22.624239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea73ec-27fb-4f0a-8086-d250a0de8669 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.631358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.631995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.632350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.633373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.634084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.636023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.666200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:22.687622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1023/2041] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:22 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:22.691032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79594fa6-ff05-4468-bf29-81664e2c767a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.692051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.784383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82e057-ce98-4fb7-b752-b694bc5fc47c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.790965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.791673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.791892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.792169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] PATH_INFO: `/v3/users/7b3df765a48d4a23be6d989a77cfd0af` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.793208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.793337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3df765a48d4a23be6d989a77cfd0af)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.794447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.795025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.795410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.795653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.796943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.800753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.846626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1024/2042] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:22 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:22.851224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.865795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e3ea0-7845-42fb-8065-dac2237dd3c4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.873278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.874245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.874557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.874839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.875739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.877542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.895766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1025/2043] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:22.896897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.909244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ff3c7-09ac-466e-b69d-de0707f27314 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:22.914737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:22.916819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:22.918254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.918536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.918725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3df765a48d4a23be6d989a77cfd0af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:22.918891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] PATH_INFO: `/v3/projects/06fa1ea3396e444099d5b410025d1701/users/bc2188b6c0754a4d9bb1e722608078d6/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:22.920481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3df765a48d4a23be6d989a77cfd0af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:22.943685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fa1ea3396e444099d5b410025d1701, user_id=bc2188b6c0754a4d9bb1e722608078d6, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:22.945795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3df765a48d4a23be6d989a77cfd0af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:22.946611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:22.962163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3df765a48d4a23be6d989a77cfd0af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:22.963295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8807e82e-74af-480b-9e22-819de865f664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3df765a48d4a23be6d989a77cfd0af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:22.970666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1026/2044] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:22 2022] PUT /identity/v3/projects/06fa1ea3396e444099d5b410025d1701/users/bc2188b6c0754a4d9bb1e722608078d6/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:22.975276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:22.977732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:22.981219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1019/2045] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:22 2022] DELETE /identity/v3/users/7b3df765a48d4a23be6d989a77cfd0af => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:22.985586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:22.986113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:22.986711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.011682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:23.015224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:23.077816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19254094-3208-4727-ab63-4bc6fb5fb359 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:23.106918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1020/2046] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:23.120160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:23.122536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:23.173236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:23.192347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:23.193104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.193255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.193521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.194229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:23.202261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:23.222728 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1027/2047] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:23.228160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:23.235113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16daa4-b5d3-4fe5-9f46-c58133e880c0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:23.240379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:23.241012 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.241878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.243373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.255525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:23.257063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45445ff3-d5f0-4d44-a248-b869b87da0cb service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:23.271072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1021/2048] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:23.272392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afa964-e1cb-4fd5-8222-d070312abaed admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:23.289239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:23.290546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.291170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.291684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] PATH_INFO: `/v3/projects/06fa1ea3396e444099d5b410025d1701/users/bc2188b6c0754a4d9bb1e722608078d6/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.295334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fa1ea3396e444099d5b410025d1701, user_id=bc2188b6c0754a4d9bb1e722608078d6, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:23.302360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:23.337689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1028/2049] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:23 2022] PUT /identity/v3/projects/06fa1ea3396e444099d5b410025d1701/users/bc2188b6c0754a4d9bb1e722608078d6/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:23.355028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.355675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.355905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.392210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] MFA Rules not processed for user `bc2188b6c0754a4d9bb1e722608078d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:23.393528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:23.483991 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:23.527133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1022/2050] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:23.542537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:23.547257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:23.671091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27d7e0-7c6c-412b-a13c-01525b1878d4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:23.676406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:23.677047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.678487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.679180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.696006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:23.698837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:23.725628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1029/2051] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:23.741218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:23.743656 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:23.869574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4225cb-f780-40d1-be19-2acf8f57c0bc None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:23.884356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:23.886258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:23.886829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:23.887171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] PATH_INFO: `/v3/projects/a299dd53e9dd41f6ae5e30fadb0f1d50` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:23.897311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a299dd53e9dd41f6ae5e30fadb0f1d50)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:23.901381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:24.019667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a299dd53e9dd41f6ae5e30fadb0f1d50', 'request_id': 'req-89804846-66c6-4ea8-a97f-9b34ae360e74'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.031343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a299dd53e9dd41f6ae5e30fadb0f1d50', 'request_id': 'req-89804846-66c6-4ea8-a97f-9b34ae360e74'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.043249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a299dd53e9dd41f6ae5e30fadb0f1d50', 'request_id': 'req-89804846-66c6-4ea8-a97f-9b34ae360e74'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.089706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] The token cache is being invalidate because project a299dd53e9dd41f6ae5e30fadb0f1d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:24.089706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.089706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.089706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1023/2052] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:23 2022] DELETE /identity/v3/projects/a299dd53e9dd41f6ae5e30fadb0f1d50 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:24.098524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:24.111382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc5505-5c20-4729-8a78-ba79ccf240b0 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:24.190573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:24.191218 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:24.191493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:24.191764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] PATH_INFO: `/v3/users/b46c34eabbf7495291f28428103ce612` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:24.193013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46c34eabbf7495291f28428103ce612)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:24.201436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:24.278383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46c34eabbf7495291f28428103ce612'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.287834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46c34eabbf7495291f28428103ce612'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.308213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46c34eabbf7495291f28428103ce612'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.308954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46c34eabbf7495291f28428103ce612'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.317126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-c61dc4f0-c03b-45de-8542-8cdfeb49d638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46c34eabbf7495291f28428103ce612'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.324757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1030/2053] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:24 2022] DELETE /identity/v3/users/b46c34eabbf7495291f28428103ce612 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:24.340742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:24.342380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:24.365099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89804846-66c6-4ea8-a97f-9b34ae360e74 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:24.460502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:24.461414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:24.461733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:24.462117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] PATH_INFO: `/v3/users/4224d4be170041b89a8057838d99887b` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:24.463398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4224d4be170041b89a8057838d99887b)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:24.465604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:24.511446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:24.556696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4224d4be170041b89a8057838d99887b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.557412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4224d4be170041b89a8057838d99887b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.577264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4224d4be170041b89a8057838d99887b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.590585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4224d4be170041b89a8057838d99887b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.591191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4224d4be170041b89a8057838d99887b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:24.612442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1024/2054] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:24 2022] DELETE /identity/v3/users/4224d4be170041b89a8057838d99887b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:24.614159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:24.614945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:24.615412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:24.624052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:24.650246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:24.651640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:24.770155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:24.773344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:24.810684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1031/2055] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:24.850440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:24.858377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:24.935840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e25a8-104c-4322-b9a5-9a47cedaa975 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:24.955118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:24.955869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:24.956220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:24.956577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] PATH_INFO: `/v3/projects/fe9a05b18dff4485be0e1c9433acc4b9` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:24.958811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9a05b18dff4485be0e1c9433acc4b9)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:24.962719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:25.015648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb44c28-d949-4bd9-9e69-69f535579863 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:25.026525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:25.027435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:25.028046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:25.029115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:25.044996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:25.046954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:25.064691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9a05b18dff4485be0e1c9433acc4b9', 'request_id': 'req-f583bbbe-1c78-4130-91d3-a571b2ddea17'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.074101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1032/2056] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:25.077684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9a05b18dff4485be0e1c9433acc4b9', 'request_id': 'req-f583bbbe-1c78-4130-91d3-a571b2ddea17'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.087974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9a05b18dff4485be0e1c9433acc4b9', 'request_id': 'req-f583bbbe-1c78-4130-91d3-a571b2ddea17'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.119150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] The token cache is being invalidate because project fe9a05b18dff4485be0e1c9433acc4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:25.122475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.123546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.126399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1025/2057] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:24 2022] DELETE /identity/v3/projects/fe9a05b18dff4485be0e1c9433acc4b9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:25.130477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:25.147240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0fddb-9e16-4399-9bd4-dd5625074fe8 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:25.277601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:25.277967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:25.278382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:25.278751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] PATH_INFO: `/v3/users/7d87b81ec4d24a9785070e5822316daa` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:25.280165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d87b81ec4d24a9785070e5822316daa)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:25.282941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:25.357326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d87b81ec4d24a9785070e5822316daa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.359404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:25.373124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d87b81ec4d24a9785070e5822316daa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.397748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d87b81ec4d24a9785070e5822316daa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.399032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d87b81ec4d24a9785070e5822316daa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.420750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d87b81ec4d24a9785070e5822316daa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.429274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1033/2058] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:25 2022] DELETE /identity/v3/users/7d87b81ec4d24a9785070e5822316daa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:25.684334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:25.685893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:25.705474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583bbbe-1c78-4130-91d3-a571b2ddea17 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:25.748993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:25.790446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:25.791142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:25.791537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:25.791793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] PATH_INFO: `/v3/projects/8e354abdb1804c8cb360087909d86cb7` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:25.792933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e354abdb1804c8cb360087909d86cb7)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:25.795062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:25.861949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e354abdb1804c8cb360087909d86cb7', 'request_id': 'req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.874393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e354abdb1804c8cb360087909d86cb7', 'request_id': 'req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.882822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e354abdb1804c8cb360087909d86cb7', 'request_id': 'req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.897123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] The token cache is being invalidate because project 8e354abdb1804c8cb360087909d86cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:25.897350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.898787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:25.903550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1026/2059] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:25 2022] DELETE /identity/v3/projects/8e354abdb1804c8cb360087909d86cb7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:25.905333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:25.907183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:25.925138 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae98e-fc6e-4278-b298-5ccc9ff975d0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:26.006659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:26.007786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.009128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.009128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] PATH_INFO: `/v3/users/bc2188b6c0754a4d9bb1e722608078d6` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.009866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2188b6c0754a4d9bb1e722608078d6)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:26.011947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:26.110519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2188b6c0754a4d9bb1e722608078d6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.124527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2188b6c0754a4d9bb1e722608078d6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.163316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2188b6c0754a4d9bb1e722608078d6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.164645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2188b6c0754a4d9bb1e722608078d6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.181152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b17b86a9-8cce-48f0-bb1d-4060768b9ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2188b6c0754a4d9bb1e722608078d6'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.193709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1034/2060] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:52:25 2022] DELETE /identity/v3/users/bc2188b6c0754a4d9bb1e722608078d6 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:26.240378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:26.244043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:26.275559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281dbbe-eb8c-45a0-8ba2-839b952df9b3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:26.389332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:26.389966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.390556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.390815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] PATH_INFO: `/v3/projects/dd0be304bd2e445280ba09f69efdaaf3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.392511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0be304bd2e445280ba09f69efdaaf3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:26.394756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:26.526514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0be304bd2e445280ba09f69efdaaf3', 'request_id': 'req-c9095d2d-f439-4a54-8513-77c252e390fa'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.551224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0be304bd2e445280ba09f69efdaaf3', 'request_id': 'req-c9095d2d-f439-4a54-8513-77c252e390fa'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.558079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:26.562025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0be304bd2e445280ba09f69efdaaf3', 'request_id': 'req-c9095d2d-f439-4a54-8513-77c252e390fa'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.596154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] The token cache is being invalidate because project dd0be304bd2e445280ba09f69efdaaf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:26.596618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.598313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-c9095d2d-f439-4a54-8513-77c252e390fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:26.604283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1027/2061] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:26 2022] DELETE /identity/v3/projects/dd0be304bd2e445280ba09f69efdaaf3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:26.606514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.606799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.607353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.672323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:26.676674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:26.775071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:26.818419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1035/2062] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:26.826873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.827176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.827427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.867478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:26.869458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:26.886789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:26.888857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1028/2063] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:26.889869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:26.890902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:26.905297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a684f06-b6c1-4a69-a757-8ee815ea2e41 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:26.918542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:26.920291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.921133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.921405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.922125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:26.922474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:26.924081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:26.927777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b5118-8a02-4064-92db-2c94cb5102e1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:26.937806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:26.938895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.940039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.940541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] PATH_INFO: `/v3/projects/06fa1ea3396e444099d5b410025d1701` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.942212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=06fa1ea3396e444099d5b410025d1701)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:26.947600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:26.956321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1029/2064] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:26.962639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:26.974026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56cf70-f723-4f34-894d-1d23f5bd4757 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:26.979853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:26.980491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:26.980835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:26.981111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:26.982488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:26.984544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.054537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06fa1ea3396e444099d5b410025d1701', 'request_id': 'req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:27.060566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1030/2065] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:26 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:27.065276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.066641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06fa1ea3396e444099d5b410025d1701', 'request_id': 'req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:27.083211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06fa1ea3396e444099d5b410025d1701', 'request_id': 'req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:27.103356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] The token cache is being invalidate because project 06fa1ea3396e444099d5b410025d1701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:52:27.103747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:27.104559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3d1ee5f9-d997-4809-98ab-d31118e8f5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:52:27.109373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1036/2066] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:52:26 2022] DELETE /identity/v3/projects/06fa1ea3396e444099d5b410025d1701 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:27.113517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73b24-f16a-4861-8ac1-b9168061e628 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.114908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73b24-f16a-4861-8ac1-b9168061e628 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.115207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73b24-f16a-4861-8ac1-b9168061e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.152195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-39a73b24-f16a-4861-8ac1-b9168061e628 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:27.155068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-39a73b24-f16a-4861-8ac1-b9168061e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:27.184313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e287fc4-4ba2-4570-ac60-5e44c1233758 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.192413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.193112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.193874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.193874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.194413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.197047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.272024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1031/2067] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:27.273938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.276763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:27.292370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ca24-1749-4853-b80e-75f0524a2201 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.302995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.303815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.304118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.304414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.307366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.311646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.312412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1037/2068] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:27.320651 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.321034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.321404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.332483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1032/2069] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:27.334664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.354997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:27.356342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00599dc8-dc52-4664-a1e5-eb5dcc408ec2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.356712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:27.364633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.365713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.366212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.366892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.367389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1038/2070] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:27.368153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.370989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.371471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.386817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1033/2071] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:27.390006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7f985-d969-41db-b636-3aa58a6376a3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.391622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.396136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.397237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.398065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.398521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.399535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:52:27.400250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.402129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.410074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d404a-51ef-4155-abcc-a34b0fb8cc59 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.432404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1039/2072] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:52:27.432568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.433267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.433523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.433664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.434199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] PATH_INFO: `/v3/projects/474d58de54cb4c44ab3885e9a2943a0c/users/a597decac02943339b9909d41020d8c7/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.445203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a38d69-26fb-4e1d-9bde-3acfae0d9da2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.451880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.452828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.453272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.453544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.454231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.456202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.460696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=474d58de54cb4c44ab3885e9a2943a0c, user_id=a597decac02943339b9909d41020d8c7, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.466316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3268e-541e-43c9-a512-32a0f06beb6e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.486083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1034/2073] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:27 2022] PUT /identity/v3/projects/474d58de54cb4c44ab3885e9a2943a0c/users/a597decac02943339b9909d41020d8c7/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:27.491530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.492398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.492969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.496969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1040/2074] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:27.502544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.534281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:27.535681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:27.575369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:27.620079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:27.638880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1035/2075] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:27.654802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.660260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:27.692772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9a0b-fc32-4569-a30d-72cf8d866f80 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.708940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.709582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.710519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.714549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.715268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.717312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.805962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1041/2076] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:27.808133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be429d9-df12-4eef-89a9-121441700d91 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.811666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.813721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.814095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.814410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.814764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.832193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a94f3-d908-4821-a626-05756b741778 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.839015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.839841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.840226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.840907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.841805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.843639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.846907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.848910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26626851-77e0-44b4-9c77-6ca553f37f0a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.894634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1042/2077] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:52:27.896740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1036/2078] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:27.898223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.935742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78541b-e64f-4c15-bf27-9956b5e568a4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:27.943334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.943766 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.944114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.946013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:27.947165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:27.948554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:27.949099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:27.950085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:27.953395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:27.977230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1043/2079] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:27.980306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:27.998484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] MFA Rules not processed for user `683b8a5c6d9048fe871fc563c9e45ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:27.999786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:28.004645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347100a-8b63-43b3-af86-ecdb8c3ab6b7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.013997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.014700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.015027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.015535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] PATH_INFO: `/v3/projects/e93c063a48fc49fa8fabbdc97999b089/users/a1e08e4811124671a08387239ba293cb/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.053150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93c063a48fc49fa8fabbdc97999b089, user_id=a1e08e4811124671a08387239ba293cb, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.058287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69cf9-9b18-4eb8-bfac-11f9047b7d8b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.119071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1044/2080] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:27 2022] PUT /identity/v3/projects/e93c063a48fc49fa8fabbdc97999b089/users/a1e08e4811124671a08387239ba293cb/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:28.130693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.131087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.131401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.214862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1037/2081] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:52:28.215131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] MFA Rules not processed for user `a1e08e4811124671a08387239ba293cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:28.219909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.220151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:28.226659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:28.258674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffa603-7630-49c1-8b13-e928ea84d590 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.373090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.373747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.374065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.374209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.374817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.377175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.384605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1045/2082] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:28.386037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.413862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7adc-c868-4ff8-afdc-219d3a8e2e91 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.419276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.419981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.420372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.420652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.421352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.423598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.456032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1038/2083] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:28 2022] POST /identity/v3/projects => generated 383 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:28.458491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.476360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:28.533130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1046/2084] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:52:28 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:52:28.535020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.568348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b453bd-caef-4bac-902e-e2ff858da157 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.674116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:28.722624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e6981-f75f-4d05-b956-10e99c8ec19b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.731208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.732093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.732646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.733106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.734119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.737345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.737546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.738533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.738832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.739089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.739841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.741987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.799958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1047/2085] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:28 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:28.804403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.815150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1039/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:52:28 2022] POST /identity/v3/users => generated 298 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:52:28.818620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.829239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2697c3-ab82-4104-b253-b44acd39c15a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.871203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0afe5c-5b87-476c-ba89-6a9ddbff7768 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.883296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.885146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.885479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.885788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.887639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.892577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.894057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.894598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.895090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.895265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.896306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:28.899849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:28.924593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1040/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:28.932992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.937020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1048/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:28.938045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:28.954602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3a86d-2a4a-4da7-b44e-10f10a289cc9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.961823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614154f6-a877-44dd-b740-29784c53412c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:28.965163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.966483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.967184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.967982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] PATH_INFO: `/v3/projects/bf13cf76844e42fab95b99c1e4ff266f/users/2015612033194e50aadc7026a02a0318/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.969253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:28.970220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:28.970720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:28.971068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] PATH_INFO: `/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5/users/d8294504e1e042e1b480036c96fd7c0f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:28.996536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a502f9c4d44fde845c3ac3c6dbf3f5, user_id=d8294504e1e042e1b480036c96fd7c0f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.004520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.017605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf13cf76844e42fab95b99c1e4ff266f, user_id=2015612033194e50aadc7026a02a0318, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.021832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.041875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1049/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:28 2022] PUT /identity/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5/users/d8294504e1e042e1b480036c96fd7c0f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:29.043674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.073486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1041/2090] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:28 2022] PUT /identity/v3/projects/bf13cf76844e42fab95b99c1e4ff266f/users/2015612033194e50aadc7026a02a0318/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:29.078245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.169390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b919481-797f-43ea-a318-99eec2547997 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.177967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96fd91-9fb7-4035-ae35-181430aa8f31 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.182701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.183363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.183627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.183896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.184563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.186288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.199963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.200954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.201402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.202077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.203091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.205187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.208573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1050/2091] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:29.213973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.226730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1042/2092] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:52:29.231754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.244242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f1227-6db8-4060-a159-22fd294c96db admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.263568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.264634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb30b9-cb14-4680-add1-fc2eebe1d68a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.265043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.265838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.266501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] PATH_INFO: `/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5/users/d8294504e1e042e1b480036c96fd7c0f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.269598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a502f9c4d44fde845c3ac3c6dbf3f5, user_id=d8294504e1e042e1b480036c96fd7c0f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.271524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.271954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.272261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.273201 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] PATH_INFO: `/v3/projects/bf13cf76844e42fab95b99c1e4ff266f/users/2015612033194e50aadc7026a02a0318/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.275474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf13cf76844e42fab95b99c1e4ff266f, user_id=2015612033194e50aadc7026a02a0318, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.277316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595d0f-f66c-4369-bc1d-2d900a30b21e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.278488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fef342-8d68-43ba-ba86-1604a1a6916a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.282333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:29.297783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1043/2093] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:29 2022] PUT /identity/v3/projects/bf13cf76844e42fab95b99c1e4ff266f/users/2015612033194e50aadc7026a02a0318/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:52:29.300584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.301054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.301553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.313756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1051/2094] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:52:29 2022] PUT /identity/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5/users/d8294504e1e042e1b480036c96fd7c0f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:52:29.330768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.331284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.332202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.349868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] MFA Rules not processed for user `a597decac02943339b9909d41020d8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:29.352302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:29.385978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] MFA Rules not processed for user `d8294504e1e042e1b480036c96fd7c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:29.386894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:29.500993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1044/2095] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:29.509540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:29.517242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.519348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:29.520092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1052/2096] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:29.529243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:29.532972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:29.614386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34eec4a-6fe3-4071-84e1-13dc88794ae4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.621542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.623811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.624230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.624392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.635666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.637179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f28761-0a48-409a-9f94-507cab2d421d service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.649742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1045/2097] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:29.745376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02def-9dde-407e-a455-3e7904a2f6f7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:29.762686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:29.763204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:29.763490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:29.763860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:29.856694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:29.859533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3e212-a756-4d3f-b69a-282e07f25715 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:29.885136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1053/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:32.543944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:32.545034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:32.545492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:32.603207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] MFA Rules not processed for user `a597decac02943339b9909d41020d8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:32.604659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:32.618133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1046/2099] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:36.345316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a4ee-1ca3-41f3-8ee7-e6b608a8f85e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:36.346246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a4ee-1ca3-41f3-8ee7-e6b608a8f85e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:36.346246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a4ee-1ca3-41f3-8ee7-e6b608a8f85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:36.382784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1258a4ee-1ca3-41f3-8ee7-e6b608a8f85e None None] MFA Rules not processed for user `2015612033194e50aadc7026a02a0318`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:36.384337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1258a4ee-1ca3-41f3-8ee7-e6b608a8f85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:36.502797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1054/2100] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:36.516862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:36.518416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:36.633900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8ee8c-946c-4e45-9716-92656654fb51 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:36.642710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:36.643576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:36.643963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:36.644340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:36.702757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:36.745697 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:36.747195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:36.760597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1047/2101] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:52:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:52:36.955226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a2460-6550-407c-b704-46f360b708f0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:36.955570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a2460-6550-407c-b704-46f360b708f0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:36.955876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a2460-6550-407c-b704-46f360b708f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:36.984509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e24a2460-6550-407c-b704-46f360b708f0 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:36.987010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e24a2460-6550-407c-b704-46f360b708f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:37.059933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:37.101111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1055/2102] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:52:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:37.169230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:52:37.171676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:37.339954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852b97-e733-450c-9be4-682bb069b98c service glance] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:52:37.345606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:52:37.346324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:37.346613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:37.346862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:37.358924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:52:37.360559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:52:37.379478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1048/2103] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:52:46.841754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6e7c-da97-4b18-bdf1-525800e8cca6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:52:46.842407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6e7c-da97-4b18-bdf1-525800e8cca6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:52:46.842407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6e7c-da97-4b18-bdf1-525800e8cca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:52:46.862181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-cbcf6e7c-da97-4b18-bdf1-525800e8cca6 None None] MFA Rules not processed for user `683b8a5c6d9048fe871fc563c9e45ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:52:46.863385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cbcf6e7c-da97-4b18-bdf1-525800e8cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:52:46.870752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1056/2104] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:52:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:52:47.368276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=43 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:52:56.856565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=10 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:06.227664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:06.237741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26b2c3-958f-4f55-b6a7-2e13fef70f16 service swift] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:06.241679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:06.242312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:06.242591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:06.242866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:06.255556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:06.257480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:06.269702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1049/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:53:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:53:11.589619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:11.590155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:11.590155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:11.616488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:11.617909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:11.625908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1057/2106] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:53:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:11.642310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:11.657312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524e9f-44a2-490c-ac66-9fcb428d495c service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:11.681048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:11.682369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:11.683063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:11.683420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:11.694894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:11.696525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:11.705763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1050/2107] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 11:53:21.609400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:21.701974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=12 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:29.370486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:29.372661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:29.465143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be2b89-99b7-4e2d-945b-c615bdea8660 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:29.470802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:29.474474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:29.475331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:29.475629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:29.479094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:29.494331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:29.496121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26459e-ead3-4e67-a42e-d9bc5848a157 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:29.507677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1058/2108] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:53:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:53:39.502597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=32 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:56.279772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:56.370118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a040f-4583-4a30-895a-4366b5b86f1e service swift] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:56.374842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:56.375467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:56.375744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:56.376019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] PATH_INFO: `/v3/users/a1e08e4811124671a08387239ba293cb` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:56.377183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e08e4811124671a08387239ba293cb)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:56.379153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:56.433827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e08e4811124671a08387239ba293cb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:56.434902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e08e4811124671a08387239ba293cb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:56.451736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e08e4811124671a08387239ba293cb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:56.461368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e08e4811124671a08387239ba293cb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:56.463185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e08e4811124671a08387239ba293cb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:56.480164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1051/2109] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:53:56 2022] DELETE /identity/v3/users/a1e08e4811124671a08387239ba293cb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:56.481876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:56.482367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:56.482464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:56.503116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:56.504360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:56.577868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1059/2110] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:56.589395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:56.590932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:56.675609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78e957-db29-4006-bc2d-73bcd6d948bf admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:56.680447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:56.681055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:56.681329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:56.681596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:56.693033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:56.695367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:56.708319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1052/2111] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:53:57.093158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:57.094706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:57.107893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0957fb-b87b-4c38-8f18-6d19108b6475 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:57.112688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:57.113320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:57.113594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:57.113891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] PATH_INFO: `/v3/projects/e93c063a48fc49fa8fabbdc97999b089` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:57.115010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93c063a48fc49fa8fabbdc97999b089)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:57.117813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:57.166698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93c063a48fc49fa8fabbdc97999b089', 'request_id': 'req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.172425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93c063a48fc49fa8fabbdc97999b089', 'request_id': 'req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.181248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93c063a48fc49fa8fabbdc97999b089', 'request_id': 'req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.192405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] The token cache is being invalidate because project e93c063a48fc49fa8fabbdc97999b089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:53:57.192688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.193453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.195280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1060/2112] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:53:57 2022] DELETE /identity/v3/projects/e93c063a48fc49fa8fabbdc97999b089 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:57.199271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:57.209107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bef250-87e9-4a65-a7ca-36190757e0d5 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:57.229452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:57.274351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:57.274979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:57.275257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:57.275531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] PATH_INFO: `/v3/users/d8294504e1e042e1b480036c96fd7c0f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:57.276667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8294504e1e042e1b480036c96fd7c0f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:57.278645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:57.350767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8294504e1e042e1b480036c96fd7c0f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.351644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8294504e1e042e1b480036c96fd7c0f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.363728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8294504e1e042e1b480036c96fd7c0f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.371381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8294504e1e042e1b480036c96fd7c0f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.372078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-519421d0-556d-48be-8249-8ea9bc1c2ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8294504e1e042e1b480036c96fd7c0f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:57.384678 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1053/2113] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:53:57 2022] DELETE /identity/v3/users/d8294504e1e042e1b480036c96fd7c0f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:57.987770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:57.989384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:58.004663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb0659-7f4c-4e3a-87bc-54af8a26e2bf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.092558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.093213 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.093499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.093812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] PATH_INFO: `/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.095380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a502f9c4d44fde845c3ac3c6dbf3f5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.097406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.136840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:58.175655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a502f9c4d44fde845c3ac3c6dbf3f5', 'request_id': 'req-546d1129-e617-440e-b7f0-7aec69456f96'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:58.186578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a502f9c4d44fde845c3ac3c6dbf3f5', 'request_id': 'req-546d1129-e617-440e-b7f0-7aec69456f96'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:58.203190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a502f9c4d44fde845c3ac3c6dbf3f5', 'request_id': 'req-546d1129-e617-440e-b7f0-7aec69456f96'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:58.217434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] The token cache is being invalidate because project a6a502f9c4d44fde845c3ac3c6dbf3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:53:58.217803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:58.218439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-546d1129-e617-440e-b7f0-7aec69456f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:53:58.219976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1061/2114] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:53:57 2022] DELETE /identity/v3/projects/a6a502f9c4d44fde845c3ac3c6dbf3f5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:58.225227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.225626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.226522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.265504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:58.267050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:58.338368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1054/2115] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:58.346455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.346766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.347051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.371366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:58.372583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:58.380892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1062/2116] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:58.386661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.399543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b5957-43cf-4607-9f04-a86364da8579 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.406579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.407200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.407477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.408172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.408392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:53:58.408870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.410523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.429914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1055/2117] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:53:58.431538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.449546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c8465-74d9-4d77-bf8e-1014191d72c2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.454834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.455472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.455738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.456006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.456646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.458644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.491400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1063/2118] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:53:58.498106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.512962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871d2cc-3343-4f54-bcf9-621f697a0eb5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.550417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:58.585249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.586020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.586416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.586754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.587499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.589534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.612662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1056/2119] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:53:58.618039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.634202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201b546-e751-4fc6-93f0-8d75fe41e8b8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.639034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.639655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.639941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.640468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.641128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.642915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.658678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1064/2120] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:53:58.659844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.672232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e7d0-d42c-48e3-91f5-8340ecfc1ecf admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.677102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.678313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.679252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.679523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.680610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.681848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109073b0-fe01-49ef-a483-43650ed10f93 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.694808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1057/2121] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:53:58.699108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.709375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594feb9-c9e7-4d9f-9a97-08d0d98cffd7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.714315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.714981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.715258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.715527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] PATH_INFO: `/v3/projects/a12a1032a0aa43a5a5f8c48a7246cb68/users/53e61aed3e8c480e842916bbc3863c2f/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.731876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12a1032a0aa43a5a5f8c48a7246cb68, user_id=53e61aed3e8c480e842916bbc3863c2f, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.734584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.750144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1065/2122] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:53:58 2022] PUT /identity/v3/projects/a12a1032a0aa43a5a5f8c48a7246cb68/users/53e61aed3e8c480e842916bbc3863c2f/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:58.760525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.760812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.761207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.791025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] MFA Rules not processed for user `53e61aed3e8c480e842916bbc3863c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:58.792248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:58.858186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1058/2123] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:58.862937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.926927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b262075-e6b4-4621-b556-d7a13db3915a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:58.931910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:58.932543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:58.932824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:58.933101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:58.933775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:58.935679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:58.964951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1066/2124] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:53:58.969041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:58.981100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8467fb-74c1-4499-9154-f8d653f0a808 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.038683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.039307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.039585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.039861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.040541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.042683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.064544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1059/2125] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:53:58 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:53:59.069041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.079920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc8f1d-5ef2-4da4-a27b-a485d1266d3f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.085391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.086028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.086306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.086595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.087226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.088829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.100386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1067/2126] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:53:59.105139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.115892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4653ed0-3df1-4b51-92d6-196a91b7e05a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.120514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.121116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.121402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.121683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.122403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.124006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c261e-2f95-42a7-9909-3d401eda107c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.134819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1060/2127] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:53:59.138979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.150741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196b791-d326-482f-9460-e1d50f0a25ec admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.155665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.156337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.156696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.156924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] PATH_INFO: `/v3/projects/dc8b0ce12a8b45b7b057144f30d0c6b3/users/17ebb923487844799920e477fc4f45e7/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.174778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8b0ce12a8b45b7b057144f30d0c6b3, user_id=17ebb923487844799920e477fc4f45e7, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.177583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.189134 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:59.199565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1068/2128] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:53:59 2022] PUT /identity/v3/projects/dc8b0ce12a8b45b7b057144f30d0c6b3/users/17ebb923487844799920e477fc4f45e7/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:59.206472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.206752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.207022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.239017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] MFA Rules not processed for user `17ebb923487844799920e477fc4f45e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:59.240420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:59.244703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:59.310999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1061/2129] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:59.315535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.386221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7983-5512-4fdd-9e33-d6794864c96c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.391994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.392823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.393295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.393755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.394552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.397313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.429796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1069/2130] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:53:59 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:53:59.434525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.448063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd73c-91fe-439b-8609-5988acdd41af None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.516120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.516812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.517200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.517538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.518286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.520420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.545193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1062/2131] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:53:59 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:53:59.549360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.560789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e2e25-4bf3-4042-b8a4-6e9423ba301b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.566463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.567127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.567412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.568257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.570078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.570943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.590851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1070/2132] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:53:59.595620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.612433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8c5f-3d35-4305-a24f-e3fe894ed873 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.618255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.618929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.619210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.619480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] PATH_INFO: `/v3/projects/c602d2cce67b4995ab7db157279bfca3/users/ed5e352831a7406489e272ff9e474a1a/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.638798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c602d2cce67b4995ab7db157279bfca3, user_id=ed5e352831a7406489e272ff9e474a1a, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.641773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.655594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1063/2133] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:53:59 2022] PUT /identity/v3/projects/c602d2cce67b4995ab7db157279bfca3/users/ed5e352831a7406489e272ff9e474a1a/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:59.659696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.670410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9a6fb-3f53-46a3-be3d-5b303c624704 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.726519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.727128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.727399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.727664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.728297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.729906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965613f-3d86-4da2-9759-896c4fe0da2a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.739520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1071/2134] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:53:59.744368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.755040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbf555-5ed3-445b-a2ea-82602b4e5f8a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.762038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.762038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.762038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.762038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] PATH_INFO: `/v3/projects/c602d2cce67b4995ab7db157279bfca3/users/ed5e352831a7406489e272ff9e474a1a/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.763773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c602d2cce67b4995ab7db157279bfca3, user_id=ed5e352831a7406489e272ff9e474a1a, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.766805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.778137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1064/2135] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:53:59 2022] PUT /identity/v3/projects/c602d2cce67b4995ab7db157279bfca3/users/ed5e352831a7406489e272ff9e474a1a/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:53:59.788740 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.789027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.789302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.820996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] MFA Rules not processed for user `ed5e352831a7406489e272ff9e474a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:53:59.822261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:59.887156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:53:59.901496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1072/2136] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:53:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:53:59.909189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:53:59.911105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:53:59.974994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756a819-fdba-46d6-9cef-ae978f1e8c84 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:53:59.978820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:53:59.979426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:53:59.979706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:53:59.979977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:53:59.989055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:53:59.990683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:53:59.995667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:00.000662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1065/2137] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:53:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:00.044553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:00.046005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:00.058265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31fc5f-dbbc-45e3-94d0-030a5f356ed0 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:00.061946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:00.062555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:00.062842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:00.063106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:00.123259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:00.124701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02841f6f-028c-41e1-ad0d-e0051688b067 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:00.134379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1073/2138] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:08.480146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:08.600163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd596c-5de9-4c57-b83f-44681c4b7b24 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:08.606165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:08.607238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:08.607632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:08.608016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] PATH_INFO: `/v3/users/53e61aed3e8c480e842916bbc3863c2f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:08.609411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e61aed3e8c480e842916bbc3863c2f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:08.611626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:08.673272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e61aed3e8c480e842916bbc3863c2f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:08.674062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e61aed3e8c480e842916bbc3863c2f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:08.688471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e61aed3e8c480e842916bbc3863c2f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:08.699593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e61aed3e8c480e842916bbc3863c2f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:08.700868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e61aed3e8c480e842916bbc3863c2f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:08.716209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1066/2139] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:08 2022] DELETE /identity/v3/users/53e61aed3e8c480e842916bbc3863c2f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:08.720944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:08.721453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:08.721623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:08.741952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:08.743812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:08.810376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1074/2140] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:08.821773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:08.823329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:08.903492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b29c-6f78-4f2f-9374-dd64cf9db03d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:08.907356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:08.907967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:08.908240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:08.908506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:08.918967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:08.920385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:08.930973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1067/2141] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:54:09.280859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:09.282480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:09.295556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07b360-607a-4f73-ade8-8272499778e3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:09.300438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:09.302304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:09.303769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:09.304053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] PATH_INFO: `/v3/projects/a12a1032a0aa43a5a5f8c48a7246cb68` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:09.305580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12a1032a0aa43a5a5f8c48a7246cb68)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:09.309161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:09.365396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12a1032a0aa43a5a5f8c48a7246cb68', 'request_id': 'req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.370813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12a1032a0aa43a5a5f8c48a7246cb68', 'request_id': 'req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.378642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12a1032a0aa43a5a5f8c48a7246cb68', 'request_id': 'req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.389922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] The token cache is being invalidate because project a12a1032a0aa43a5a5f8c48a7246cb68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:09.390459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.391154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.392628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1075/2142] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:09 2022] DELETE /identity/v3/projects/a12a1032a0aa43a5a5f8c48a7246cb68 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:09.397358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:09.406884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6db1ca-d56b-4373-a616-2c5b06e801d9 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:09.425180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:09.468733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:09.469364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:09.469635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:09.469939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] PATH_INFO: `/v3/users/17ebb923487844799920e477fc4f45e7` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:09.471140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ebb923487844799920e477fc4f45e7)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:09.473069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:09.518987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ebb923487844799920e477fc4f45e7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.519628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ebb923487844799920e477fc4f45e7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.532084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ebb923487844799920e477fc4f45e7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.541236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ebb923487844799920e477fc4f45e7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.541976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ebb923487844799920e477fc4f45e7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:09.554727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1068/2143] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:09 2022] DELETE /identity/v3/users/17ebb923487844799920e477fc4f45e7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:09.882290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:09.884034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:09.903671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd5b20-f5cd-43ea-94c2-10f72c69579b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:09.922764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:09.977670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:09.978498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:09.978812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:09.979133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] PATH_INFO: `/v3/projects/dc8b0ce12a8b45b7b057144f30d0c6b3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:09.981139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8b0ce12a8b45b7b057144f30d0c6b3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:09.983156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:10.034701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8b0ce12a8b45b7b057144f30d0c6b3', 'request_id': 'req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.041167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8b0ce12a8b45b7b057144f30d0c6b3', 'request_id': 'req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.048800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8b0ce12a8b45b7b057144f30d0c6b3', 'request_id': 'req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.062532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] The token cache is being invalidate because project dc8b0ce12a8b45b7b057144f30d0c6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:10.062817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.063475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.064992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1076/2144] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:09 2022] DELETE /identity/v3/projects/dc8b0ce12a8b45b7b057144f30d0c6b3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:10.070080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:10.071633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:10.085585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabb3a1-c11e-4e2b-ab30-17ef560bd5c5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:10.151290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:10.151944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:10.152216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:10.152507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] PATH_INFO: `/v3/users/ed5e352831a7406489e272ff9e474a1a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:10.153624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5e352831a7406489e272ff9e474a1a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:10.155631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:10.201494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:10.204534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5e352831a7406489e272ff9e474a1a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.205742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5e352831a7406489e272ff9e474a1a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.219284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5e352831a7406489e272ff9e474a1a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.227605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5e352831a7406489e272ff9e474a1a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.228235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc858cc4-d706-416a-9a98-50b2eba5bdf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5e352831a7406489e272ff9e474a1a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.240610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1069/2145] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:10 2022] DELETE /identity/v3/users/ed5e352831a7406489e272ff9e474a1a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:10.609681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:10.611725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:10.653469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8f45-b40e-42e1-ab94-bd3a3d7966d1 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:10.737102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:10.738316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:10.738872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:10.739338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] PATH_INFO: `/v3/projects/c602d2cce67b4995ab7db157279bfca3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:10.741342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c602d2cce67b4995ab7db157279bfca3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:10.743624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:10.826090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c602d2cce67b4995ab7db157279bfca3', 'request_id': 'req-835257cb-880d-4b49-b77c-9f85509c4f74'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.828092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:10.838589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c602d2cce67b4995ab7db157279bfca3', 'request_id': 'req-835257cb-880d-4b49-b77c-9f85509c4f74'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.850561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c602d2cce67b4995ab7db157279bfca3', 'request_id': 'req-835257cb-880d-4b49-b77c-9f85509c4f74'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.872092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] The token cache is being invalidate because project c602d2cce67b4995ab7db157279bfca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:10.872395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.874212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-835257cb-880d-4b49-b77c-9f85509c4f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:10.876114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1077/2146] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:10 2022] DELETE /identity/v3/projects/c602d2cce67b4995ab7db157279bfca3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:10.883252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:10.885589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:10.886292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:10.919835 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:10.924654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:11.018205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1070/2147] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:11.020466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.021529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.021529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:11.043569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:11.053936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1078/2148] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:11.058003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.068467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca584f6-01ce-40e7-a888-e597c04d79b5 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.073473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.074257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.074534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.074808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.075470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:54:11.075776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.077403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.095790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.099747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1071/2149] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:54:11.105943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba8dde-673b-47b2-8c03-0669229d551a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.110984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.111604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.111885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.112156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.112985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.115611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.145406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1079/2150] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:11 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:54:11.151090 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.164816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca59f-d26d-4e1a-a135-072d13df5716 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.229612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.230257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.230544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.230819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.231464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.233402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.259596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1072/2151] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:11 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 11:54:11.263263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.274173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f1ba-002d-4770-9ea2-2c33eea5c70e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.278861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.279478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.279758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.280032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.280680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.282348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.293277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1080/2152] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:11.298614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.309417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001aef4-45c4-4e2c-8773-080a1682b7cb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.314197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.314863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.315086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.315360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.316010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.317625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aef47-abbe-4ccb-8d32-9c1e0207da91 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.333989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1073/2153] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:11.338590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.349510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fd13a-d2a6-4dfe-afa4-ed2652fa870b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.354235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.354891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.355170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.355446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] PATH_INFO: `/v3/projects/63f9d608da8341cc98d474d53108d0e7/users/0ffa34e38d684677b1776b7e441828b9/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.372824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f9d608da8341cc98d474d53108d0e7, user_id=0ffa34e38d684677b1776b7e441828b9, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.375602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.391107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1081/2154] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:11 2022] PUT /identity/v3/projects/63f9d608da8341cc98d474d53108d0e7/users/0ffa34e38d684677b1776b7e441828b9/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:11.402739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.403052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.403326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.432691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] MFA Rules not processed for user `0ffa34e38d684677b1776b7e441828b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:11.433971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:11.447144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:11.502908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1074/2155] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:11.715622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:11.717491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:11.779731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936f11-1102-45d5-884a-6fa8a81d9529 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:11.783414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:11.784057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:11.784357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:11.784629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:11.794116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:11.795535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f70ff9-c85a-4b88-9ee4-850f5dacf57f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:11.805472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1082/2156] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:14.505335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:14.507381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:14.574340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f8abd-7bd5-4294-8d94-8fc2a0a7beb9 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:14.578868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:14.579486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:14.579760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:14.580029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] PATH_INFO: `/v3/users/eca27d28e19a434eae7195f4061bff44` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:14.581139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca27d28e19a434eae7195f4061bff44)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:14.583096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:14.629668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca27d28e19a434eae7195f4061bff44'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:14.630334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca27d28e19a434eae7195f4061bff44'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:14.645190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca27d28e19a434eae7195f4061bff44'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:14.652737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca27d28e19a434eae7195f4061bff44'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:14.653351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca27d28e19a434eae7195f4061bff44'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:14.665996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1075/2157] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:14 2022] DELETE /identity/v3/users/eca27d28e19a434eae7195f4061bff44 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:14.670464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:14.670934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:14.671050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:14.692223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:14.694137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:14.751759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1083/2158] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:54:14.762054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:14.763511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:14.803905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:14.823401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02f871-6997-4ee7-80bd-f61032b3b58b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:14.827964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:14.828569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:14.828850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:14.829127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:14.839211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:14.840675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538478e2-11ba-4c54-825a-15e13af8d55d service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:14.850539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1076/2159] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:15.256781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.262708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:15.272689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49b22-b0a6-47ec-a01e-890d6053163f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.277212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.277963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.278304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.278640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] PATH_INFO: `/v3/projects/24b14fcbaeb647e79ec038c4c5cfe013` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.280114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b14fcbaeb647e79ec038c4c5cfe013)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.282184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.299162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:15.350893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b14fcbaeb647e79ec038c4c5cfe013', 'request_id': 'req-6d068e16-f5fd-439d-9976-f5b97fe51a4d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:15.358745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b14fcbaeb647e79ec038c4c5cfe013', 'request_id': 'req-6d068e16-f5fd-439d-9976-f5b97fe51a4d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:15.368111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b14fcbaeb647e79ec038c4c5cfe013', 'request_id': 'req-6d068e16-f5fd-439d-9976-f5b97fe51a4d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:15.379995 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] The token cache is being invalidate because project 24b14fcbaeb647e79ec038c4c5cfe013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:15.380285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:15.381576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6d068e16-f5fd-439d-9976-f5b97fe51a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:15.383215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1084/2160] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:15 2022] DELETE /identity/v3/projects/24b14fcbaeb647e79ec038c4c5cfe013 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:15.388504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.388874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.389153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.410228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:15.411441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:15.475060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1077/2161] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:15.483685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.483968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.484241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.504900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:15.506534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:15.514461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1085/2162] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:15.519033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.532459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b14b4-b5b4-410a-baa9-1041545fabd6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.539342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.539963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.540240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.540649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.541507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:54:15.541633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.543265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.556653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1078/2163] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:54:15.561167 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.571666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9fae-8770-4e5f-a0e8-be704a1a11c2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.576402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.577004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.577282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.577553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.578222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.580144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.610597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1086/2164] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:15 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:15.614769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.626349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f139a-bab4-4623-a767-4e9abc122aee admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.685871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.686482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.686757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.687045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.687664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.689588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.720067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1079/2165] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:15 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:15.724231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.735254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12744a3-c432-4725-bb6e-1ec7fa51e48c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.739805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.740568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.740849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.741118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.741833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.743423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.753334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1087/2166] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:15.758636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.768945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571d21-538d-423d-94cd-9ea868b58081 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.774406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.775003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.775281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.775688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.776337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.778042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fdf11-0ee8-416a-861b-065709051983 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.789205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1080/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:15.793693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.805157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbad2c-0449-460f-b234-68d32b27454f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:15.810022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:15.810677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.810952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.811244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] PATH_INFO: `/v3/projects/626e605a578247499347d7f323eb3dd0/users/105841db84aa4c41808a8d7667f487af/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.828068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626e605a578247499347d7f323eb3dd0, user_id=105841db84aa4c41808a8d7667f487af, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:15.831832 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:15.848000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1088/2168] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:15 2022] PUT /identity/v3/projects/626e605a578247499347d7f323eb3dd0/users/105841db84aa4c41808a8d7667f487af/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:15.859534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:15.859816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:15.860089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:15.890796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] MFA Rules not processed for user `105841db84aa4c41808a8d7667f487af`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:15.892022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:15.926028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:15.961047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1081/2169] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:15.972606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:15.974539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:16.034933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e6e4-4476-42dc-83ac-51476da863f5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:16.038836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:16.039454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.039733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.040007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.048759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:16.050245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e869-8d5d-4640-9500-ef9fc4277864 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:16.060616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1089/2170] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:16.089439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:16.161013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6203b0e-7613-456d-8f9d-a512ccb3967b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:16.166175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:16.166878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.167158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.167458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] PATH_INFO: `/v3/users/105841db84aa4c41808a8d7667f487af` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.168627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105841db84aa4c41808a8d7667f487af)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:16.170660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:16.228253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105841db84aa4c41808a8d7667f487af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.229004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105841db84aa4c41808a8d7667f487af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.245774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105841db84aa4c41808a8d7667f487af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.254190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105841db84aa4c41808a8d7667f487af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.254820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105841db84aa4c41808a8d7667f487af'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.269660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1082/2171] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:16 2022] DELETE /identity/v3/users/105841db84aa4c41808a8d7667f487af => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:16.274350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.274641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.274923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.293994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:16.295130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:16.329552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:16.366896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1090/2172] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:16.374024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:16.375583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:16.445368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac360cf-7930-449f-9913-15d366c61167 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:16.449180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:16.449960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.450239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.450508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.458785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:16.461735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:16.463638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2f61-c512-46ec-bccf-e84d68e83607 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:16.477687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1083/2173] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:16.811304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:16.813017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:16.828001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eaa3f-f93b-4cf8-b4ea-f9828e0e7418 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:16.833574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:16.835161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.835444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.835719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] PATH_INFO: `/v3/projects/626e605a578247499347d7f323eb3dd0` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.837149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=626e605a578247499347d7f323eb3dd0)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:16.839059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:16.890831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626e605a578247499347d7f323eb3dd0', 'request_id': 'req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.899292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626e605a578247499347d7f323eb3dd0', 'request_id': 'req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.908186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626e605a578247499347d7f323eb3dd0', 'request_id': 'req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.920875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] The token cache is being invalidate because project 626e605a578247499347d7f323eb3dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:16.921164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.921857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1a00fe6e-449d-412d-a1ed-d67c2d0235c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:16.923379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1091/2174] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:16 2022] DELETE /identity/v3/projects/626e605a578247499347d7f323eb3dd0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:16.929200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:16.929744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:16.930223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:16.953344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:16.954602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:17.032125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1084/2175] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:54:17.039970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.040303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.040521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.063534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:17.064628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:17.077539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1092/2176] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:17.082146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.093323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456ee7-2ee4-4951-bfbf-0f31e1b557d4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.098143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.098751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.099030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.099906 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.100578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:54:17.100882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.103306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.116961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1085/2177] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:54:17.121726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.132851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8537b-7a22-4d0d-81cc-c8d6acf045f8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.137751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.138370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.138648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.138946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.139598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.141873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.173421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1093/2178] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:17 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:17.177926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.188554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce852602-c720-43c7-89d9-ee0eb6f3f6c7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.257009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.257731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.258018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.258306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.259424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.261433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.289406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1086/2179] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:17 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:17.293569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.305235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e97ff-03f6-4eea-962f-7e9fa5a1f4d0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.310321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.310941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.311219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.311490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.312149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.313855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.327032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1094/2180] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:17.337684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.356440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e06dda-99f5-444b-95ee-08ad0f8b7ccc admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.364293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.366143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.366928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.367418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.368620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.371624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fcc75-9357-4594-bf82-93c7bcf6cefe admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.391019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1087/2181] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:17.397300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.410615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15102-cd85-49dd-aece-181c7a4f9145 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.417712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.418539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.418819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.419091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] PATH_INFO: `/v3/projects/1a02769540e24f608519f1ff69eae9c1/users/e010b776bedc4a30aa2d341d6fd04352/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.438810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a02769540e24f608519f1ff69eae9c1, user_id=e010b776bedc4a30aa2d341d6fd04352, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.441592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.463784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1095/2182] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:17 2022] PUT /identity/v3/projects/1a02769540e24f608519f1ff69eae9c1/users/e010b776bedc4a30aa2d341d6fd04352/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:17.475404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.475791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.476220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.517908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] MFA Rules not processed for user `e010b776bedc4a30aa2d341d6fd04352`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:17.519155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:17.576215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:17.599327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1088/2183] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:17.607363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:17.609157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:17.674531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1370a-d7b0-40be-b968-cf8bd9c82480 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:17.683623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:17.684256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:17.684537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:17.684806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:17.694649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:17.696154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efedb4-8e01-41d7-9c77-5d057ea8bd93 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:17.705861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1096/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:27.591844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:27.701973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:31.831989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:31.833715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:31.904893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4ff7a-86cc-410c-aeb2-b8ea6dc17352 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:31.908898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:31.909538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:31.909840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:31.910108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:31.912010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:32.009580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:32.011484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:32.028481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1089/2185] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:54:32.779750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bfb12-f5f5-49cf-83ac-c5c26602978b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:32.780372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bfb12-f5f5-49cf-83ac-c5c26602978b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:32.780372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bfb12-f5f5-49cf-83ac-c5c26602978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:32.803296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-328bfb12-f5f5-49cf-83ac-c5c26602978b None None] MFA Rules not processed for user `a597decac02943339b9909d41020d8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:32.804556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-328bfb12-f5f5-49cf-83ac-c5c26602978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:32.937887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1097/2186] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:33.223676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:33.315725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91456-3220-4b21-b6d8-92ba3c5926ee None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:33.318189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91456-3220-4b21-b6d8-92ba3c5926ee None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:33.319373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91456-3220-4b21-b6d8-92ba3c5926ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:33.345139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-53d91456-3220-4b21-b6d8-92ba3c5926ee None None] MFA Rules not processed for user `41bc35497fe94ababe0c824a429168d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:33.346768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-53d91456-3220-4b21-b6d8-92ba3c5926ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:33.410935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff6713-c1ec-429e-8557-1c293acadc7a service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:33.425843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:33.426777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:33.427213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:33.428230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] PATH_INFO: `/v3/users/e010b776bedc4a30aa2d341d6fd04352` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:33.431117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1098/2187] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:33.441904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e010b776bedc4a30aa2d341d6fd04352)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:33.446082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:33.528836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e010b776bedc4a30aa2d341d6fd04352'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:33.530170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e010b776bedc4a30aa2d341d6fd04352'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:33.538955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:33.549440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e010b776bedc4a30aa2d341d6fd04352'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:33.563674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e010b776bedc4a30aa2d341d6fd04352'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:33.565212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e010b776bedc4a30aa2d341d6fd04352'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:33.589717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1090/2188] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:33 2022] DELETE /identity/v3/users/e010b776bedc4a30aa2d341d6fd04352 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:33.596049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:33.596356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:33.596635 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:33.621728 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:33.622956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:33.699199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1099/2189] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:33.709231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:33.711034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:33.786299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15350298-7489-485d-bf22-eab5efe44986 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:33.791485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:33.792243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:33.792627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:33.793004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:33.805098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:33.807275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:33.830084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1091/2190] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:34.238380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:34.241147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:34.255038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e1df5-5c0d-4092-a011-9c1f64ce6470 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:34.266939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:34.268406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:34.269044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:34.269501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] PATH_INFO: `/v3/projects/1a02769540e24f608519f1ff69eae9c1` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:34.272337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a02769540e24f608519f1ff69eae9c1)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:34.274750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:34.358884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:34.367347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a02769540e24f608519f1ff69eae9c1', 'request_id': 'req-2f085d43-fbb9-409e-9e56-99a85bee1bc8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:34.378767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a02769540e24f608519f1ff69eae9c1', 'request_id': 'req-2f085d43-fbb9-409e-9e56-99a85bee1bc8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:34.391586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a02769540e24f608519f1ff69eae9c1', 'request_id': 'req-2f085d43-fbb9-409e-9e56-99a85bee1bc8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:34.410322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] The token cache is being invalidate because project 1a02769540e24f608519f1ff69eae9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:34.410751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:34.411685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2f085d43-fbb9-409e-9e56-99a85bee1bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:34.429697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1100/2191] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:34 2022] DELETE /identity/v3/projects/1a02769540e24f608519f1ff69eae9c1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:43.815383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:DELETE=3,keystone:SELECT=36,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:44.103918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:44.105576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:44.121818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfb627-181f-42ef-b7c1-e40b23b2c91a service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:44.195945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:44.196616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:44.196896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:44.197167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] PATH_INFO: `/v3/users/0ffa34e38d684677b1776b7e441828b9` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:44.198526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffa34e38d684677b1776b7e441828b9)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:44.200501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:44.269219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffa34e38d684677b1776b7e441828b9'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.270077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffa34e38d684677b1776b7e441828b9'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.290410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffa34e38d684677b1776b7e441828b9'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.305092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffa34e38d684677b1776b7e441828b9'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.305765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffa34e38d684677b1776b7e441828b9'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.320278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1092/2192] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:44 2022] DELETE /identity/v3/users/0ffa34e38d684677b1776b7e441828b9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:44.325047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:44.325718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:44.325718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:44.348055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:44.350209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:44.437111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1101/2193] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:54:44.448102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:44.450220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:44.546126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c58-d1f9-4bee-a9e4-24340727e3c5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:44.550434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:44.551206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:44.551668 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:44.552057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:44.563595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:44.565070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c12d0-0f10-4c57-9254-08721fb95cc8 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:44.577682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1093/2194] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:44.885442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:44.887112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:44.902021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbc33c-8d45-4562-9af7-c448b3e09d82 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:44.912360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:44.913757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:44.914232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:44.914611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] PATH_INFO: `/v3/projects/63f9d608da8341cc98d474d53108d0e7` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:44.916925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f9d608da8341cc98d474d53108d0e7)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:44.919499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:44.987719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f9d608da8341cc98d474d53108d0e7', 'request_id': 'req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:44.996638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f9d608da8341cc98d474d53108d0e7', 'request_id': 'req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:45.011585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f9d608da8341cc98d474d53108d0e7', 'request_id': 'req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:45.035722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] The token cache is being invalidate because project 63f9d608da8341cc98d474d53108d0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:45.036226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:45.037253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8e6f0b8a-5d2d-4473-a722-5d48430abbbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:45.040239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1102/2195] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:44 2022] DELETE /identity/v3/projects/63f9d608da8341cc98d474d53108d0e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:45.048331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.048960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.049262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.073204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:45.074644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:45.096488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:45.176366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1094/2196] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:45.184586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.184860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.185155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.212736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:45.214077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:45.222447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1103/2197] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:45.226968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.237267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b5428-1857-4634-930d-48f8ae44dee5 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.242672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.243299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.243575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.243842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.244515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:54:45.244816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.246992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.260932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1095/2198] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 11:54:45.265742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.275947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f1e3a-8b0f-4579-97de-6cb441e7750a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.281069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.281736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.282016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.282283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.282975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.284931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.317700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1104/2199] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:45.323117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.334288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828c29a-df98-44f7-a102-71f694c39bba admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.402859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.403470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.403746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.404010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.404654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.406645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.430384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1096/2200] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:45.434561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.446292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b490eb-7850-4163-ba2a-d2d680fffdef admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.451776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.452417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.452696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.452969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.453917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.455512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.471552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1105/2201] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:45.473485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.493723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236acd-6070-4a47-822b-5e8d050ddd33 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.503085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.504170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.504681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.505193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.506453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.510300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfac0d-2943-4268-a001-ebcbffd7538f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.530586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1097/2202] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:45.535316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.546852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095356-358d-48b1-8ac9-6dc84de980e7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.552159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.552841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.553105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.554456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] PATH_INFO: `/v3/projects/1128d8d2d1254cdaa24b880214930f3a/users/a6c05da578af4e4a870711af8995dbd1/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.578652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1128d8d2d1254cdaa24b880214930f3a, user_id=a6c05da578af4e4a870711af8995dbd1, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.581540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.584306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:45.599697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1106/2203] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:45 2022] PUT /identity/v3/projects/1128d8d2d1254cdaa24b880214930f3a/users/a6c05da578af4e4a870711af8995dbd1/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:45.612132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.612448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.612663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.646701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] MFA Rules not processed for user `a6c05da578af4e4a870711af8995dbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:45.648042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:45.753318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1098/2204] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:45.754553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.835443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17f2f7-acc8-476c-8b19-472d50af7941 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.840374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.840993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.841362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.841531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.842261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.844193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.876520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1107/2205] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:45.881294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:45.895064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98590dcb-2a2e-4f66-aa5f-424af2f31596 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:45.909559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:45.958502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:45.959120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:45.959407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:45.960257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:45.960902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:45.962914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:45.986924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1099/2206] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:45 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:45.993497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.007160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9527bc-dae6-491d-be2c-10443e8181e5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.011922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.012551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.012823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.013091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.013785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.015681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.028794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1108/2207] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:46.033094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.045138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68114e-4cd6-47c6-92b5-59bae7baa570 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.050187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.050817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.051065 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.051337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.051987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.053777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27e6d2-7a49-4ee5-a02e-2c19fd71314e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.064508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1100/2208] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:46.068870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.086104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4e4a2-9892-4a23-bb15-458f3f6892b2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.093509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.094622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.095180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.095602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] PATH_INFO: `/v3/projects/c873fb2dbdb245dfb17629ffa76b2fba/users/eb53e4317e644de39ea1e68de9ea1590/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.117535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c873fb2dbdb245dfb17629ffa76b2fba, user_id=eb53e4317e644de39ea1e68de9ea1590, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.120620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.135610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1109/2209] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:46 2022] PUT /identity/v3/projects/c873fb2dbdb245dfb17629ffa76b2fba/users/eb53e4317e644de39ea1e68de9ea1590/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:46.146500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.146790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.147068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.180344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] MFA Rules not processed for user `eb53e4317e644de39ea1e68de9ea1590`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:46.181607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:46.270584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1101/2210] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:46.275073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.350920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dce36-04fa-4983-9fdb-a5b115416a9c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.357308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.358267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.358585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.358875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.359533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.361756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.399695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1110/2211] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:46 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:46.404313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.416706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79ea2-17f0-4884-8a81-caa52b676eda None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.505317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.506045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.506348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.507150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.507800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.509802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.537403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1102/2212] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:46 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:46.542558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.557226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315bbee-2601-4466-9537-bf2119fd5803 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.565453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.566111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.566385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.566652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.567309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.568964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.581494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1111/2213] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:54:46.586812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.597569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddae2c9-949e-4131-a7d0-01e88f5fd8b0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.603144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.604114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.604585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.604993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] PATH_INFO: `/v3/projects/2d2e87e4a96f4792881b15b6a591af01/users/fdc4708e02ca4ccbaa3189bc340b2817/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.621021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:46.624581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2e87e4a96f4792881b15b6a591af01, user_id=fdc4708e02ca4ccbaa3189bc340b2817, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.628427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.644889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1103/2214] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:46 2022] PUT /identity/v3/projects/2d2e87e4a96f4792881b15b6a591af01/users/fdc4708e02ca4ccbaa3189bc340b2817/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:46.649831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.664210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7d558-277f-45ea-ba82-7107f575397c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.684796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:46.747577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.748295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.748674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.749494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.750027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.751739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7318e1-e7c2-4030-bd31-383566e33c6a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.764078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1112/2215] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:46.768341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:46.784011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d097a-aed7-4f5c-bffb-08f0709473ce admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:46.792686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:46.793161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.793347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.793706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] PATH_INFO: `/v3/projects/2d2e87e4a96f4792881b15b6a591af01/users/fdc4708e02ca4ccbaa3189bc340b2817/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.796635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2e87e4a96f4792881b15b6a591af01, user_id=fdc4708e02ca4ccbaa3189bc340b2817, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:46.800587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:46.836451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1104/2216] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:46 2022] PUT /identity/v3/projects/2d2e87e4a96f4792881b15b6a591af01/users/fdc4708e02ca4ccbaa3189bc340b2817/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:46.838663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:46.838939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:46.839216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:46.903870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] MFA Rules not processed for user `fdc4708e02ca4ccbaa3189bc340b2817`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:46.906831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:47.050496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1113/2217] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:54:47.064796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:47.067630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:47.173046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b455d-0229-48c6-a14e-39b033bab646 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:47.183412 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:47.184154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:47.184511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:47.184847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:47.290091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:47.292197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:47.308043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1105/2218] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:49.046927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:49.048681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:49.061448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b76bc-7ffb-479d-ad3b-088ee1e42932 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:49.065348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:49.066009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:49.066280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:49.066548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:49.129582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:49.131054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:49.141596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1114/2219] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:54:49.749056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:49.758910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c305be4-95b5-4514-8838-2541b7a95dad service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:49.762599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:49.763220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:49.763486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:49.763750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:49.774191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:49.775650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721df4a6-1082-40f0-9a9e-672984e5f5d4 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:49.785681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1106/2220] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:53.715091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:53.757256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:53.782256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45627d26-3051-4f25-ac01-13e272f9ba0e service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:53.787129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:53.788520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:53.788801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:53.789075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] PATH_INFO: `/v3/users/a6c05da578af4e4a870711af8995dbd1` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:53.790251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c05da578af4e4a870711af8995dbd1)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:53.792203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:53.841402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c05da578af4e4a870711af8995dbd1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:53.848116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c05da578af4e4a870711af8995dbd1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:53.861079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c05da578af4e4a870711af8995dbd1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:53.861773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c05da578af4e4a870711af8995dbd1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:53.868904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c05da578af4e4a870711af8995dbd1'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:53.875117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1115/2221] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:53 2022] DELETE /identity/v3/users/a6c05da578af4e4a870711af8995dbd1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:53.880479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:53.880967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:53.881087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:53.901873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:53.903093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:53.962923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1107/2222] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:54:53.972612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:53.974049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:54.037343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fb109-e58b-4c40-83b2-52b944e306fa admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:54.040993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:54.041601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:54.041915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:54.042185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:54.051267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:54.052698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:54.062691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1116/2223] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:54.457794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:54.460061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:54.472782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb16d7-dd8a-4b08-aca7-e3e75d8c37e8 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:54.479706 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:54.480365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:54.480654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:54.480921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] PATH_INFO: `/v3/projects/1128d8d2d1254cdaa24b880214930f3a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:54.482122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1128d8d2d1254cdaa24b880214930f3a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:54.484081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:54.503481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:54.543933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1128d8d2d1254cdaa24b880214930f3a', 'request_id': 'req-6c74c39c-6a57-491e-a398-6849978b7673'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.551961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1128d8d2d1254cdaa24b880214930f3a', 'request_id': 'req-6c74c39c-6a57-491e-a398-6849978b7673'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.558867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1128d8d2d1254cdaa24b880214930f3a', 'request_id': 'req-6c74c39c-6a57-491e-a398-6849978b7673'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.570432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] The token cache is being invalidate because project 1128d8d2d1254cdaa24b880214930f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:54.570719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.571386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.572878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1108/2224] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:54 2022] DELETE /identity/v3/projects/1128d8d2d1254cdaa24b880214930f3a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:54:54.576847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:54.587311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e82b-2442-42d2-8db0-d8445fee3c70 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:54.652301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:54.652967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:54.653227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:54.653500 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] PATH_INFO: `/v3/users/eb53e4317e644de39ea1e68de9ea1590` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:54.654654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb53e4317e644de39ea1e68de9ea1590)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:54.656603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:54.658397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:54.708302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb53e4317e644de39ea1e68de9ea1590'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.717434 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb53e4317e644de39ea1e68de9ea1590'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.730127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb53e4317e644de39ea1e68de9ea1590'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.730812 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb53e4317e644de39ea1e68de9ea1590'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.741391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb53e4317e644de39ea1e68de9ea1590'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:54.747636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1117/2225] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:54 2022] DELETE /identity/v3/users/eb53e4317e644de39ea1e68de9ea1590 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:55.043048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:55.046373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:55.061184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74c39c-6a57-491e-a398-6849978b7673 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:55.128191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:55.128826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:55.129116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:55.129387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] PATH_INFO: `/v3/projects/c873fb2dbdb245dfb17629ffa76b2fba` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:55.130555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c873fb2dbdb245dfb17629ffa76b2fba)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:55.133035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:55.186129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c873fb2dbdb245dfb17629ffa76b2fba', 'request_id': 'req-7a3a8633-48b6-4031-bdda-485df9be2c7d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.195528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c873fb2dbdb245dfb17629ffa76b2fba', 'request_id': 'req-7a3a8633-48b6-4031-bdda-485df9be2c7d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.201936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c873fb2dbdb245dfb17629ffa76b2fba', 'request_id': 'req-7a3a8633-48b6-4031-bdda-485df9be2c7d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.223253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] The token cache is being invalidate because project c873fb2dbdb245dfb17629ffa76b2fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:55.223707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.227855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.229535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1109/2226] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:55 2022] DELETE /identity/v3/projects/c873fb2dbdb245dfb17629ffa76b2fba => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:55.233836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:55.235266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:55.249333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b07c0-c235-4a93-a1af-5e466d62a489 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:55.310784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:55.311472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:55.311719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:55.311960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] PATH_INFO: `/v3/users/fdc4708e02ca4ccbaa3189bc340b2817` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:55.313073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc4708e02ca4ccbaa3189bc340b2817)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:55.315082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:55.362133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc4708e02ca4ccbaa3189bc340b2817'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.369792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc4708e02ca4ccbaa3189bc340b2817'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.381796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc4708e02ca4ccbaa3189bc340b2817'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.382569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc4708e02ca4ccbaa3189bc340b2817'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.389347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5666c313-f135-4639-a5cb-10f19cb4dcee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc4708e02ca4ccbaa3189bc340b2817'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:55.391021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:55.396917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1118/2227] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:54:55 2022] DELETE /identity/v3/users/fdc4708e02ca4ccbaa3189bc340b2817 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:55.833053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:55.836777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:55.850676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a8633-48b6-4031-bdda-485df9be2c7d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:55.911026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:55.941973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:55.942627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:55.942903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:55.943173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] PATH_INFO: `/v3/projects/2d2e87e4a96f4792881b15b6a591af01` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:55.944998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2e87e4a96f4792881b15b6a591af01)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:55.946960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:55.996073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2e87e4a96f4792881b15b6a591af01', 'request_id': 'req-1ee89a42-e76d-4dad-8694-f74034212f7c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:56.006185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2e87e4a96f4792881b15b6a591af01', 'request_id': 'req-1ee89a42-e76d-4dad-8694-f74034212f7c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:56.014745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2e87e4a96f4792881b15b6a591af01', 'request_id': 'req-1ee89a42-e76d-4dad-8694-f74034212f7c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:56.029831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] The token cache is being invalidate because project 2d2e87e4a96f4792881b15b6a591af01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:54:56.030124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:56.030857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1ee89a42-e76d-4dad-8694-f74034212f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:54:56.032369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1110/2228] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:54:55 2022] DELETE /identity/v3/projects/2d2e87e4a96f4792881b15b6a591af01 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:56.038767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.039303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.039402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.060891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:56.062193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:56.124770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1119/2229] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:56.132754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.133261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.133353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.153335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:56.154582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:56.163543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1111/2230] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:56.167933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.177478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace790-9a40-42fa-a538-7179bf93ec47 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.182795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.183391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.183665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.184260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.184907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:54:56.185208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.186879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.198990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1120/2231] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:54:56.203383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.212917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5cfde-71dc-4b43-90d8-b89adb6c412b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.217674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.218294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.218574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.218844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.219471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.221379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.250749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1112/2232] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:54:56 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:56.254757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.265833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610802fb-76d4-41e7-890e-b94e45df23ea admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.324769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.325375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.325674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.325955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.326594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.328526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.354100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1121/2233] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:54:56 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:54:56.358401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.369473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562387ad-19c3-4c17-bf03-d00340fd9b22 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.374224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.374891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.375105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.375376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.376021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.377626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.387792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1113/2234] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:56.392102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.402830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59ca4b-44f3-4171-85e0-249761c05ed7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.407659 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.408264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.408543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.408816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.409464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.411102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805a516-b7f4-486f-a894-e9e737cfc2a7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.423621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1122/2235] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 11:54:56.427520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.438382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb314121-6e21-42bb-913d-891a8469f5d1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.444261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.444781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.445079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.445349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] PATH_INFO: `/v3/projects/223f7e0f4ea74b398aad606cf1e7c150/users/317022dc9f1d46cbbe7190edd7832e43/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.462207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223f7e0f4ea74b398aad606cf1e7c150, user_id=317022dc9f1d46cbbe7190edd7832e43, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.464654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.478174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1114/2236] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:54:56 2022] PUT /identity/v3/projects/223f7e0f4ea74b398aad606cf1e7c150/users/317022dc9f1d46cbbe7190edd7832e43/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:54:56.483158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2dac-5f8f-4b4d-9cc4-1f0f62b478e9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.483442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2dac-5f8f-4b4d-9cc4-1f0f62b478e9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.483708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2dac-5f8f-4b4d-9cc4-1f0f62b478e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.503380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6c7e2dac-5f8f-4b4d-9cc4-1f0f62b478e9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:56.504945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6c7e2dac-5f8f-4b4d-9cc4-1f0f62b478e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:56.565973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1123/2237] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:56.577311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:56.579043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:56.638510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076805a-db0d-4866-812a-68fb070bfcb4 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:56.642430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:56.643042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:56.643318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:56.643588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:56.654252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:56.655717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:56.665967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1115/2238] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:54:57.456319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc669b5-c515-4d4f-b00a-d67cc4945741 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:57.456835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc669b5-c515-4d4f-b00a-d67cc4945741 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:57.456835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc669b5-c515-4d4f-b00a-d67cc4945741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:57.465474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:57.488711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4bc669b5-c515-4d4f-b00a-d67cc4945741 None None] MFA Rules not processed for user `317022dc9f1d46cbbe7190edd7832e43`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:54:57.490305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4bc669b5-c515-4d4f-b00a-d67cc4945741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:57.559720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1124/2239] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:54:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:54:57.572043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:54:57.573482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:54:57.578746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:54:57.650647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a41c-a192-43f3-8f59-75c45136027a service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:54:57.654568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:54:57.655254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:54:57.655537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:54:57.655808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:54:57.664761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:54:57.666309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1adb6-8596-4933-8f99-21510c651a23 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:54:57.675299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1116/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:54:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:00.207431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:00.208047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:00.208047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:00.233536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] MFA Rules not processed for user `317022dc9f1d46cbbe7190edd7832e43`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:00.234539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:00.243950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1125/2241] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:55:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:55:07.671267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=25 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:08.879762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3303-b453-428b-86ba-db4fd5873f7d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:08.880345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3303-b453-428b-86ba-db4fd5873f7d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:08.880345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3303-b453-428b-86ba-db4fd5873f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:08.901713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-322a3303-b453-428b-86ba-db4fd5873f7d None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:08.903222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-322a3303-b453-428b-86ba-db4fd5873f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:08.960206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1117/2242] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:55:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:55:09.001155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:09.003179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:09.076871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c52e8a-128f-4877-97c3-71f0504ab272 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:09.083083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:09.084523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:09.084919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:09.085198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:09.094000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:09.095606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:09.104762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1126/2243] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:55:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:55:18.952601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=25 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:19.101864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=65 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:27.656202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:27.656774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:27.656774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:27.678357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:27.679553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:27.687230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1118/2244] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 6 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 11:55:27.701503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:27.712527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bbe98-1fd7-4871-9405-71c0c6a186f9 service swift] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:27.717539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:27.718202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:27.718443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:27.718709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:27.727816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:27.729396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:27.737665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1127/2245] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:55:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:55:29.708373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:29.709869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:29.725129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fb437-ff85-481a-8e2e-ae2a73689ed4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:29.731051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:29.731681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:29.731956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:29.732371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] PATH_INFO: `/v3/users/a597decac02943339b9909d41020d8c7` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:29.733385 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a597decac02943339b9909d41020d8c7)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:29.735361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:29.783660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a597decac02943339b9909d41020d8c7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:29.784301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a597decac02943339b9909d41020d8c7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:29.797088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a597decac02943339b9909d41020d8c7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:29.805096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a597decac02943339b9909d41020d8c7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:29.805829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a5158c81-ad61-4ebd-8cc2-5f77a7d11ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a597decac02943339b9909d41020d8c7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:29.819380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1119/2246] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:55:29 2022] DELETE /identity/v3/users/a597decac02943339b9909d41020d8c7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:55:29.998805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.019664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486e67-fc78-4e12-9ac1-59ff2cec46c6 service swift] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.081026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.081602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.081915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.082187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] PATH_INFO: `/v3/projects/474d58de54cb4c44ab3885e9a2943a0c` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.083308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474d58de54cb4c44ab3885e9a2943a0c)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.085899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.135220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474d58de54cb4c44ab3885e9a2943a0c', 'request_id': 'req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:30.142221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474d58de54cb4c44ab3885e9a2943a0c', 'request_id': 'req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:30.150573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474d58de54cb4c44ab3885e9a2943a0c', 'request_id': 'req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:30.161990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] The token cache is being invalidate because project 474d58de54cb4c44ab3885e9a2943a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:55:30.162282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:30.162915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9091ad06-31e9-4f36-9c10-a6e93acb8cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:55:30.164873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1128/2247] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:55:29 2022] DELETE /identity/v3/projects/474d58de54cb4c44ab3885e9a2943a0c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:55:30.169461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.169786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.170069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.198593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:30.199831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.273727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1120/2248] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:55:30.281488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.281802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.282074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.301935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:30.303144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.312225 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1129/2249] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:55:30.316642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.328020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cb915-edc4-4f10-9640-4f69e47dc788 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.332993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.333615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.333831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.334105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.334761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 11:55:30.335113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.336743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.348578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1121/2250] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 11:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 11:55:30.354338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.363987 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985380b-d6e3-443f-bf6a-f33bc81dfb6e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.369783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.370392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.370671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.370943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.371607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.373527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.402197 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1130/2251] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 11:55:30 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:55:30.406522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.418162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb20137-c366-4c70-88a8-b443cb8756f0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.421480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.423550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.485071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:30.485322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:30.496964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.498039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.498654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.499069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.499774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.501987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.514205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e170-c912-4878-8357-02e4f755402d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.522576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.524526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.524807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.525112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.527365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.532124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1122/2252] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 11:55:30 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 11:55:30.536298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.547311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47078a72-c4e3-4742-8a0c-adc04c630876 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.552855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.553452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.553754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.554050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.554690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.556314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.567137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1123/2253] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:55:30.572095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.581970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446ad34-b2c4-4ab1-b825-812cb6e1d965 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.587883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.588415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.588885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.589011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.589602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.591219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.592423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.594246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5d07-a787-4f02-b5fd-ad30daded21c service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.603484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1124/2254] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 11:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 11:55:30.604788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1131/2255] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:55:30.607995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.620430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b4460-da3f-4e38-87fa-53cdb053e0b9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.625146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.625852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.626133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.626436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] PATH_INFO: `/v3/projects/33c6fd80d1e441e8afd6a78b8e121e09/users/2bf72c1d7789430babc068858f97ea0e/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.645761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c6fd80d1e441e8afd6a78b8e121e09, user_id=2bf72c1d7789430babc068858f97ea0e, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.649527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.663297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1125/2256] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 11:55:30 2022] PUT /identity/v3/projects/33c6fd80d1e441e8afd6a78b8e121e09/users/2bf72c1d7789430babc068858f97ea0e/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 11:55:30.734772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.734772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.734772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.765460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] MFA Rules not processed for user `2bf72c1d7789430babc068858f97ea0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:55:30.766850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.839367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1132/2257] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:55:30.853179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:30.854961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:30.936174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4774-c6ab-4fdf-8ef2-e21df903091e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:30.940383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:30.940383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:30.940979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:30.941263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:30.953330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:30.954866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:30.966747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1126/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:33.098604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:33.100282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:33.184664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78181d-7c99-45c5-b333-2e2a379524f6 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:33.190922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:33.191565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:33.191847 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:33.192146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:33.194731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:33.206194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:33.266899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:33.268408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:33.278126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1133/2259] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:33.887657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:33.889070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:33.953914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272a0d-f6b3-42ae-8bfa-e3d1511a0341 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:33.957747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:33.958352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:33.958628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:33.958897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:33.960705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:33.977149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:33.979284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:33.991081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1127/2260] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:34.181877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:34.183416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:34.250694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899b59-4290-4ead-90d9-fd6e3dae5389 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:34.254766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:34.255392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:34.256244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:34.256244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:34.258352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:34.272816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:34.274549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:34.285577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1134/2261] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:55:37.611363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:37.622635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4df57-a155-49c2-b54f-c746a726de2b service glance] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:37.628182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:37.628807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:37.629082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:37.629350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:37.631151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:37.674189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:37.694293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:37.696488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:37.706738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1128/2262] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:44.280767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=48 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:47.702304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=9 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:55:47.900450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:55:47.910517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917aae6-9373-4fa3-9a17-b74c1e5a7721 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:55:47.914248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:55:47.914895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:55:47.915179 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:55:47.915687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:55:47.919735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:55:47.931622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:55:47.933751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4686-87a8-4993-9374-1a70a87677e9 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:55:47.947631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1135/2263] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:55:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:55:57.942441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:56:00.777236 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:00.841985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8901-7290-4148-ae8e-61f4a8eb5204 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:00.846824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:00.847443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:00.847724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:00.847999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] PATH_INFO: `/v3/users/2bf72c1d7789430babc068858f97ea0e` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:00.849129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf72c1d7789430babc068858f97ea0e)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:00.851092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:00.907653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf72c1d7789430babc068858f97ea0e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:00.908724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf72c1d7789430babc068858f97ea0e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:00.921283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf72c1d7789430babc068858f97ea0e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:00.929017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf72c1d7789430babc068858f97ea0e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:00.929680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf72c1d7789430babc068858f97ea0e'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:00.942973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1129/2264] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 11:56:00 2022] DELETE /identity/v3/users/2bf72c1d7789430babc068858f97ea0e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:56:00.948035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:00.948536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:00.948536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:00.968518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:00.969759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:01.036173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1136/2265] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:01.045919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:01.047341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:01.111313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968db5f-be42-46bb-bfb2-4b7e2e4604bb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:01.115610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:01.116221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:01.116792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:01.117079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:01.127156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:01.128589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f58e03-3efb-4fe7-bd3a-2b0b570be8da service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:01.138328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1130/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:01.407083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:01.409049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:01.421054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874b82-8e82-4a7b-afb1-1873db639164 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:01.426455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:01.427149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:01.427428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:01.427727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] PATH_INFO: `/v3/projects/33c6fd80d1e441e8afd6a78b8e121e09` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:01.428942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c6fd80d1e441e8afd6a78b8e121e09)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:01.430942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:01.479912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c6fd80d1e441e8afd6a78b8e121e09', 'request_id': 'req-91940082-6aa8-4797-9edf-ab34946887f5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:01.485469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c6fd80d1e441e8afd6a78b8e121e09', 'request_id': 'req-91940082-6aa8-4797-9edf-ab34946887f5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:01.494992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c6fd80d1e441e8afd6a78b8e121e09', 'request_id': 'req-91940082-6aa8-4797-9edf-ab34946887f5'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:01.507760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] The token cache is being invalidate because project 33c6fd80d1e441e8afd6a78b8e121e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 11:56:01.508073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:01.508738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 11:56:01.510619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1137/2267] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 11:56:01 2022] DELETE /identity/v3/projects/33c6fd80d1e441e8afd6a78b8e121e09 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 11:56:05.161030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:05.161664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:05.161664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:05.183538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] MFA Rules not processed for user `e05967546ef245d2b88874f6cf3f84fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:05.184789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:05.204047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:56:05.258813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1131/2268] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:05.587763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:05.589783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:05.663547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91940082-6aa8-4797-9edf-ab34946887f5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:05.667848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:05.668463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:05.668745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:05.669020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:05.670855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:05.681589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:05.683096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:05.693108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1138/2269] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:06.037166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:06.038854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:06.118470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbe4f3-524b-4044-8e77-ba9a42c8d2ec None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:06.123720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:06.124307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:06.124587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:06.124857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:06.127252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:06.140005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:06.141622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:06.153290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1132/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:09.675254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:09.676806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:09.713546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:56:09.781485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0f30-ae98-41bb-8aa6-2ffa7788a980 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:09.788046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:09.788722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:09.788945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:09.789223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:09.791629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:09.799435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:09.801190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:09.808604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:09.810388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23458da-5020-4843-b5f1-bd78e21ba70f service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:09.819036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f57c1-4887-43ab-8ad2-b099cdb32448 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:09.823460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1139/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:09.827567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:09.828842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:09.830988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:09.830988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:09.833863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:09.852145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:09.854110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:09.868914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1133/2272] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:56:18.455285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f877-a8b2-4b5a-9bac-447d73206bb1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:18.455851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f877-a8b2-4b5a-9bac-447d73206bb1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:18.455851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f877-a8b2-4b5a-9bac-447d73206bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:18.485989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e635f877-a8b2-4b5a-9bac-447d73206bb1 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:18.488346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e635f877-a8b2-4b5a-9bac-447d73206bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:18.558013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1140/2273] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:18.589354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:18.591161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:18.684931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34470-f66f-4f61-a819-84c7e6e54ed3 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:18.690987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:18.691595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:18.691869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:18.692139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:18.701275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:18.702932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ff61-949f-45c0-a2df-26163c670a9a service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:18.712917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1134/2274] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:56:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:56:28.550381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=47 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:56:28.709234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=85 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:56:42.024001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.024613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.024613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.025917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1141/2275] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:56:42.033188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccde840-7414-4dfd-98b4-16249a5f79fd None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.033793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccde840-7414-4dfd-98b4-16249a5f79fd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.033984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccde840-7414-4dfd-98b4-16249a5f79fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.056822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0ccde840-7414-4dfd-98b4-16249a5f79fd None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:42.058183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0ccde840-7414-4dfd-98b4-16249a5f79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:42.071720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1135/2276] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 11:56:42.089242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:42.091022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:42.160623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce3291-d4b5-408b-9fef-973b59e26609 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:42.164343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:42.168295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.168537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.168813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.182083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:42.183537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e82e86-4b17-4174-93c9-9f0562eed008 service glance] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:42.196885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1142/2277] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:42.288314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.289002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.289476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] PATH_INFO: `/` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.291534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1136/2278] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:56:42.299519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc64f2-6cd5-4823-a67e-d0dcda5a670c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.299913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc64f2-6cd5-4823-a67e-d0dcda5a670c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.300253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc64f2-6cd5-4823-a67e-d0dcda5a670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.322593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-29dc64f2-6cd5-4823-a67e-d0dcda5a670c None None] MFA Rules not processed for user `b52624a6d02e43aeb9bb33e61f1bfac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:42.323681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-29dc64f2-6cd5-4823-a67e-d0dcda5a670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:42.333240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1143/2279] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:42.346410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:42.348122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:42.365467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f8716-b7fa-465d-9c9d-3899aa780cdd None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:42.370231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:42.370841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.371123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.371394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.379467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:42.380953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4833d7-4c2d-4bc3-b31f-39865d7ae803 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:42.389108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1137/2280] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:56:42.477337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546da6-e5a8-4b56-970f-98f18b48ff06 None None] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.477751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546da6-e5a8-4b56-970f-98f18b48ff06 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.478031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546da6-e5a8-4b56-970f-98f18b48ff06 None None] PATH_INFO: `/` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.479382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1144/2281] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 6 11:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 11:56:42.485299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c40d5-55c5-4e19-948b-6c2c605dd885 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.485672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c40d5-55c5-4e19-948b-6c2c605dd885 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.485978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c40d5-55c5-4e19-948b-6c2c605dd885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.507631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-137c40d5-55c5-4e19-948b-6c2c605dd885 None None] MFA Rules not processed for user `9260fd9f9c524d20a9025595a53ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:42.508924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-137c40d5-55c5-4e19-948b-6c2c605dd885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:42.521248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1138/2282] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 6 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:42.533185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2546da6-e5a8-4b56-970f-98f18b48ff06 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:42.547779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2546da6-e5a8-4b56-970f-98f18b48ff06 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:42.552349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:42.553051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:42.553960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:42.554515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:42.568752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:42.571002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:42.584016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1145/2283] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 11:56:49.660262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:49.660825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:49.661262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:49.684646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:56:49.686099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:49.695077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1139/2284] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 11:56:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:56:49.708785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:49.722139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878af9-c403-4336-95cc-63cb035a5ad2 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:49.728042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:49.728730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:49.729076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:49.729429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:49.739353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:49.741044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830052d0-83a2-45de-aead-a8d452029380 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:49.751157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1146/2285] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:56:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 11:56:50.324736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:56:50.342007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf9605-aecb-417b-b726-14e79f432cf6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:56:50.347859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:56:50.348476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:56:50.348755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:56:50.349018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:56:50.350817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:56:50.363575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:56:50.365193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0046d3-e7e8-4e27-89ed-c6eed8e987b7 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:56:50.376570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1140/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:56:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:56:59.746257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=41 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:57:00.371383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=21 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:57:22.545972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:57:22.547003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:57:22.547618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:57:22.571388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] MFA Rules not processed for user `317022dc9f1d46cbbe7190edd7832e43`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:57:22.572761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:57:22.637717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1147/2287] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:57:27.704461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:57:27.704987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:57:27.705092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:57:27.724966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:57:27.726299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:57:27.733607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1141/2288] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 6 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:57:27.756889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:57:27.770478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253712f7-2fe7-4981-85f5-204c58b246aa None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:57:27.775637 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:57:27.776244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:57:27.776525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:57:27.776791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:57:27.788244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:57:27.789980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:57:27.807462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1148/2289] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 11:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 11:57:37.719818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:57:37.795259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=31 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:57:51.718100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:57:51.731962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5156a6-0cc3-45e7-a280-46a039007899 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:57:51.738698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:57:51.739311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:57:51.739588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:57:51.739857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:57:51.741622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:57:51.807568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:57:51.809162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:57:51.818394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1142/2290] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:57:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:58:01.814074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=28 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:59:02.686446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:59:02.688153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:02.820703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0008c-31a8-429a-8ca9-237ab577836b service swift] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:59:02.826993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:59:02.827886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:59:02.828163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:59:02.828629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:59:02.830510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:02.846396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:59:02.848150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e26c48-1eee-4eb7-bb64-7a43bfd635ef service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:59:02.860704 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1149/2291] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:59:12.856264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=37 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:59:13.801781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:59:13.861362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be3f9-55d7-49de-a098-0d3731e6d2ea service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:59:13.866847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:59:13.867441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:59:13.867714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:59:13.867977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:59:13.869736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:13.929134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:59:13.930769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:59:13.940913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1143/2292] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 11:59:23.935737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=49 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:59:34.477914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:59:34.478441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:59:34.478441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:59:34.498629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] MFA Rules not processed for user `41bc35497fe94ababe0c824a429168d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 11:59:34.499871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:34.507185 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1150/2293] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 11:59:44.492604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 11:59:57.817417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 11:59:57.819141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:57.903311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2fd-79a6-40ea-aa42-245384137d35 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 11:59:57.907313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 11:59:57.907930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 11:59:57.908206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 11:59:57.908482 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 11:59:57.910977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 11:59:57.923053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 11:59:57.924628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 11:59:57.935284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1144/2294] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 11:59:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:00:07.930423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=35 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:00:30.972301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:00:30.985385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98411b-1cc2-483e-a335-845d92a8a99b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:00:30.997928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:00:30.998677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:00:30.998953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:00:30.999222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:00:31.001274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:00:31.095562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:00:31.097163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:00:31.116180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1151/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:00:33.954108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:00:33.967277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697227a-759a-4d07-bf0c-511f313d152a service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:00:33.972684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:00:33.973273 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:00:33.973546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:00:33.973842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:00:33.975625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:00:34.009015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:00:34.010666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:00:34.031725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1145/2296] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:00:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:00:38.121538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:00:38.130881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae94e8-fa48-4d91-904f-3bf8888fa79b service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:00:38.135079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:00:38.135675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:00:38.135952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:00:38.136221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:00:38.137992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:00:38.221476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:00:38.222933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:00:38.242016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1152/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:00:44.028405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=19 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:00:47.529571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:00:47.550049 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190389d-13bd-4859-9b86-b79dac346ecc service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:00:47.554085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:00:47.554705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:00:47.554985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:00:47.555259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:00:47.557247 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:00:47.584061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:00:47.585502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:00:47.597106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1146/2298] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:00:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:00:48.240029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=82 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:00:57.592234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=11 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:01:15.028549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:01:15.050717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604ce47-5427-4bf4-892d-7d1dbb335a9f service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:01:15.054474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:01:15.055084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:01:15.055358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:01:15.055623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:01:15.057495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:01:15.084591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:01:15.086077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:01:15.096020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1153/2299] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:01:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:01:25.092155 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=15 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:02:51.489575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:02:51.501959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e244ae-bab3-4709-8458-9f3bc03b4891 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:02:51.508560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:02:51.509355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:02:51.509635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:02:51.509932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:02:51.518593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:02:51.520426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:02:51.548930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1147/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:02:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:03:01.546369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=13 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:04:12.385144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:04:12.397709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845e004-d59b-438c-8e77-60b8256f5d33 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:04:12.402907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:04:12.403517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:04:12.403801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:04:12.404074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:04:12.413458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:04:12.415119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:04:12.426443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1154/2301] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:04:14.234560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:04:14.247377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3fec-ccd8-402b-b54f-95126a20d426 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:04:14.261520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:04:14.262184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:04:14.262458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:04:14.262727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:04:14.271831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:04:14.273433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:04:14.306135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1148/2302] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:04:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:04:22.422097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:04:24.302172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=13 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:04:57.938612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:04:57.947939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c51d-92e8-457a-9ac6-1e51593f0c97 service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:04:57.951564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:04:57.952156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:04:57.952435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:04:57.952707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:04:57.961440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:04:57.963522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:04:57.991541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1155/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:04:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:05:07.988894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=13 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:05:31.973059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:05:31.974501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:05:31.986959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f247e-541e-4843-bfff-14192aa15cac service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:05:31.990625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:05:31.991221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:05:31.991500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:05:31.991770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:05:32.000461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:05:32.001965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:05:32.011614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1149/2304] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:05:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:05:33.581278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:05:33.590752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccaf99-3918-4cdb-9ce5-76e8d32ed7e2 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:05:33.594413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:05:33.595005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:05:33.595282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:05:33.595549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:05:33.604860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:05:33.606311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:05:33.615172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1156/2305] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:05:39.226250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:05:39.248172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0174b6-0c6e-4783-9818-8cd42318697f service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:05:39.253840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:05:39.254434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:05:39.254703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:05:39.254971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:05:39.264083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:05:39.265710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:05:39.275855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1150/2306] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:05:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:05:43.611114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:05:49.270633 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=12 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:06:20.031438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:06:20.032939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:06:20.095088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d3a49-d369-4e20-8f2d-bd86e3b52998 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:06:20.098938 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:06:20.099636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:06:20.100438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:06:20.100719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:06:20.108686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:06:20.110163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:06:20.118420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1157/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:06:30.114815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=26 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:07:51.349973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:07:51.360231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668eac-e88b-472e-9eef-ac95d784a962 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:07:51.363873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:07:51.364464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:07:51.364742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:07:51.365014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:07:51.374246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:07:51.375680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:07:51.385211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1151/2308] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:08:01.380636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:09:14.282570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:09:14.303295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896759-00bc-4957-999b-3435f0c7c744 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:09:14.307111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:09:14.307710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:09:14.307986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:09:14.308252 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:09:14.310136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:09:14.410671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:09:14.412244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:09:14.422705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1158/2309] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:09:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:09:21.343074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:09:21.344490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:09:21.476135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98e1a5-da80-417f-9215-689c2e67ee85 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:09:21.481252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944357-2569-45b3-8982-c600288980ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:09:21.481909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944357-2569-45b3-8982-c600288980ba service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:09:21.482191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944357-2569-45b3-8982-c600288980ba service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:09:21.482467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944357-2569-45b3-8982-c600288980ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:09:21.485533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c8944357-2569-45b3-8982-c600288980ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:09:21.497123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8944357-2569-45b3-8982-c600288980ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:09:21.498698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8944357-2569-45b3-8982-c600288980ba service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:09:21.509212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1152/2310] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:09:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:09:24.417452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=48 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:09:31.505005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=59 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:09:57.120848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:09:57.130614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5d0f-8ce7-4488-b483-dd04277c1f8f service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:09:57.134226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:09:57.134831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:09:57.135113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:09:57.135385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:09:57.210209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:09:57.211568 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:09:57.232481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1159/2311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:09:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:09:58.375217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944357-2569-45b3-8982-c600288980ba service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:09:58.376789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c8944357-2569-45b3-8982-c600288980ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:09:58.394380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944357-2569-45b3-8982-c600288980ba service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:09:58.400846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:09:58.401149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:09:58.401501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:09:58.401821 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:09:58.403731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:09:58.484879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:09:58.486542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:09:58.496246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1153/2312] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:09:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:10:07.229865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=41 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:10:08.492212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=33 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:10:32.297310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:10:32.298989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:32.390917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96e0ff-e6ec-499e-9a89-6ea56330911c service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:10:32.395449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:10:32.396165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:10:32.396448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:10:32.396723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:10:32.398722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:32.506387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:10:32.507870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:10:32.528806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1160/2313] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:10:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:10:34.078803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:10:34.089360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f121f-03c3-4b67-9b92-f55b8c1c0f2b service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:10:34.093000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:10:34.093603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:10:34.093911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:10:34.094184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:10:34.096186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:34.212810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:10:34.217325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:10:34.243080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1154/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:10:38.507058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:10:38.508677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:38.526115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849b1cf-8418-4fa5-b5c7-1c4deb9b90c2 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:10:38.531863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:10:38.532623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:10:38.532739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:10:38.533009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:10:38.534999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:38.606407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:10:38.621477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:10:38.623202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:10:38.644248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1161/2315] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:10:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:10:44.238028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=41 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:10:44.895185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:10:44.897151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:44.909312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a689926-948c-4488-a293-fe7c61b8e82d service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:10:44.913950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:10:44.914585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:10:44.914891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:10:44.915159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:10:44.916982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:10:44.933102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:10:44.934586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:10:44.945619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1155/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:10:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:10:48.640931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=11 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:10:54.940682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:11:27.011216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:11:27.033127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0570a2-8186-44e3-82a8-2ac36237b495 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:11:27.036761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:11:27.037373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:11:27.037974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:11:27.037974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:11:27.039915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:11:27.063834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:11:27.065336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:11:27.074233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1162/2317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:11:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:11:37.070584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=15 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:12:51.730403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:12:51.741973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f1848-87fe-4428-af48-a0563ccb9edf service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:12:51.747438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:12:51.748027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:12:51.748308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:12:51.749461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:12:51.751520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:12:51.833491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:12:51.835047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:12:51.854197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1156/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:12:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:13:01.850990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=41 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:14:15.191880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:14:15.205638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26922da-a1fb-428d-9076-f1560b7a339b service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:14:15.211508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:14:15.212371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:14:15.212580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:14:15.212915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:14:15.223324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:14:15.225020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:14:15.259664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1163/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:14:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:14:25.255496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=13 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:14:27.352212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:14:27.364059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01caad-82b8-418e-8a52-8c7d9962f981 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:14:27.369270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:14:27.369901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:14:27.370818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:14:27.371022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:14:27.382734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:14:27.388106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:14:27.414920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1157/2320] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:14:37.405929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:14:55.589328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:14:55.599492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f87fb-14c8-4561-931f-6f38433713f9 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:14:55.603163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:14:55.603748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:14:55.604057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:14:55.604323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:14:55.606243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:14:55.634565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:14:55.636009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:14:55.649118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1164/2321] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:14:58.456616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:14:58.469353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2973ba-1714-4a54-8042-b2cc7f089f81 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:14:58.475643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:14:58.476327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:14:58.476675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:14:58.477019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:14:58.485400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:14:58.487036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:14:58.516183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1158/2322] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:15:05.642424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=11 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:15:08.513036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=13 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:15:32.500232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:15:32.515469 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c3596-dbfd-4446-93b8-a03e4f8e3fa3 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:15:32.521455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:15:32.522147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:15:32.522430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:15:32.522697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:15:32.532005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:15:32.533629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:15:32.544243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1165/2323] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:15:34.461000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:15:34.475508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0732e5-f2d0-49db-9112-e5095d731747 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:15:34.484763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:15:34.485909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:15:34.486193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:15:34.486486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:15:34.497692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:15:34.500212 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:15:34.512607 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1159/2324] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:15:38.886732 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:15:38.888519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:15:38.902431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c396-0faa-433f-b901-7fa901fde293 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:15:38.907129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:15:38.907950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:15:38.908232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:15:38.908502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:15:38.918297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:15:38.919780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:15:38.930506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1166/2325] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:15:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:15:44.507855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:15:45.332506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:15:45.342725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a364-c7b8-4c90-9c58-4b70b15e52a6 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:15:45.346291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:15:45.346919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:15:45.347199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:15:45.347463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:15:45.358795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:15:45.360234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:15:45.374116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1160/2326] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:15:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:15:48.925372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=12 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:15:55.368849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:16:32.175526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:16:32.177588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:16:32.270050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b7c87-83f8-46a4-9704-2dab62ddcff8 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:16:32.276366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:16:32.277602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:16:32.278182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:16:32.278670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:16:32.290365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:16:32.292566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac20c-d5c2-4e97-bce3-9b7987c1eddb service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:16:32.304152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1167/2327] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:16:42.298314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=26 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:16:45.928224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:16:45.944456 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0d974-8a43-4899-bfba-6aaab34423c3 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:16:45.951422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:16:45.953073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:16:45.953963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:16:45.954465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:16:45.957457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:16:45.989372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:16:45.991920 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:16:46.022005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1161/2328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:16:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:16:56.000305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:11.352915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:11.353494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:11.353494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:11.381064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] MFA Rules not processed for user `e05967546ef245d2b88874f6cf3f84fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:11.382942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:11.389964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1168/2329] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:11.711219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:11.721103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa7b4-271f-4220-a4ed-38f48c04513b service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:11.724888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:11.726088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:11.726366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:11.726636 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:11.728490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:11.739208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:11.740867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:11.749562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1162/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:13.114186 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:13.127224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a6047-cc3d-4f10-8e24-20272e8de07a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:13.132902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:13.133524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:13.133842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:13.134119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:13.136018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:13.149014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:13.150664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80a4a5-0fb6-4b33-8be6-f8c50e74ef10 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:13.162286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1169/2331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:16.406654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:16.432475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f1f5-0cf5-4c5a-8de9-1b00c0fe55f3 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:16.448821 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:16.450487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:16.451191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:16.451874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:16.455268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:16.481418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:16.485307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:16.526722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1163/2332] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:17.607816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:17.608366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:17.608366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:17.725355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:17.731507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:17.856842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1170/2333] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 6 12:17:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:17:17.919220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:17.920819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:18.023084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d332d83-aa40-490b-a1a0-b82876c95e03 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:18.029276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:18.029824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:18.030307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:18.030722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:18.042510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:18.044157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65e54e-d0ac-42d5-9b2f-fffdad48a4d3 service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:18.056545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1164/2334] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 12:17:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 12:17:18.716358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:18.730402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44039-a26f-4e07-b2d0-3e6191e06cbc None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:18.734774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:18.735664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:18.735955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:18.736251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:18.738680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:18.858731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:18.860268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05853ecb-eb63-4a36-b848-d62efabaae68 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:18.882381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1171/2335] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:19.160683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead39dc-4b69-416e-9dca-ac6017c25c18 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:19.161488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead39dc-4b69-416e-9dca-ac6017c25c18 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:19.161488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead39dc-4b69-416e-9dca-ac6017c25c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:19.199229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-6ead39dc-4b69-416e-9dca-ac6017c25c18 None None] MFA Rules not processed for user `e235718ad1da42ebb209fb4b13626ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:19.200572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6ead39dc-4b69-416e-9dca-ac6017c25c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:19.208468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1165/2336] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:22.252217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.252656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.252816 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.277766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:22.324570 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:22.325857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:22.335079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1172/2337] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:22.343643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.345927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.346574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.371465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:22.372617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:22.389088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1166/2338] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:22.390711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.404962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d273807-e2c4-44a1-88cd-eec6130beec6 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.413275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.415366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.415928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.416385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.417461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:17:22.417921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.419742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.434620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1173/2339] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:17:22.439261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.451443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326052d-19b9-40ee-8deb-1c49939b7ec3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.456695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.457352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.457629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.457942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.458587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.461469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.522462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1167/2340] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:17:22 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:17:22.527172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.545392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be09f5e-2888-4448-8c8a-fcb920debf54 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.625601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.626360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.626735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.627102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.628481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.630052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.660548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1174/2341] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:17:22 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:17:22.664785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.686904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f31c2-3ad8-4fa3-9b67-b219339c77dc admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.692587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.693230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.693507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.694345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.695398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.697268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.714045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1168/2342] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:17:22.719869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.737721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f78d40-8f08-44da-aef2-491ea2717762 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.748136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.749205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.750021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.750735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.752046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.755217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cacf9a-69f9-4d6e-91b1-e12d969e0f22 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.778189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1175/2343] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:17:22.782623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:22.801070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9c09c-a93d-4f92-af4a-1070e1758c26 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:22.807167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:22.808122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.808500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.808871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] PATH_INFO: `/v3/projects/2006ceb6c9d641019ec86aea7e4b676f/users/6d920e2427d145feb7ae253d5eda28b7/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.837972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2006ceb6c9d641019ec86aea7e4b676f, user_id=6d920e2427d145feb7ae253d5eda28b7, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:22.840842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:22.876713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1169/2344] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:17:22 2022] PUT /identity/v3/projects/2006ceb6c9d641019ec86aea7e4b676f/users/6d920e2427d145feb7ae253d5eda28b7/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:17:22.879930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:22.880318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:22.881198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:22.907912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:22.909164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:22.992833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1176/2345] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:23.005416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:23.039343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:23.096401 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0b79-8114-49ff-96e2-4b11f8f74a55 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:23.102605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:23.103627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:23.104048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:23.104595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:23.122716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:23.124999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac07cd-1984-453d-bb39-cd47d3e7f8d6 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:23.142171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1170/2346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:17:25.392837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:25.437748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:25.514504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fc639-465d-47f5-8d96-bc0401a0c8fb None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:25.523240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:25.523549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:25.523777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:25.524059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:25.526194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:25.541725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:25.544093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:25.557959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1177/2347] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:29.407434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:29.408041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:29.408481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:29.468387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] MFA Rules not processed for user `6d920e2427d145feb7ae253d5eda28b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:29.469583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:29.574828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1171/2348] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:29.582012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:29.604799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3968-d277-46d5-980f-ad594e881b3c service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:29.618675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:29.619442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:29.619822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:29.620163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:29.620999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:29.623086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:29.663004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1178/2349] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:17:29 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:17:29.667182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:29.679434 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7731-ce95-4ce8-b511-d2afb459dc2e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:29.801286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:29.801807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:29.802092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:29.804604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:29.805590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:29.808896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:29.846318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1172/2350] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:17:29 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:17:29.848244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:29.865691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c45cc-e1bd-4ce3-b419-598b5705cf30 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:29.878195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:29.878970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:29.879285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:29.880181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:29.880389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:29.883687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:29.905352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1179/2351] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:17:29.906524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:29.924144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b866-921e-4720-869b-2fbdd9857dc3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:29.938183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:29.940620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:29.940620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:29.940620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] PATH_INFO: `/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd/users/388f279266084960967414017ebd9fcd/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:29.962756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f6ce9c64cd413986df3a0a2ac956cd, user_id=388f279266084960967414017ebd9fcd, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:29.978034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:30.010805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1173/2352] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:17:29 2022] PUT /identity/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd/users/388f279266084960967414017ebd9fcd/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:17:30.012966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:30.026039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93874-497c-442e-81f9-5276ae3f9d63 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:30.126932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:30.128083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:30.129339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:30.134599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:30.135646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:30.137751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379e83a-a843-4e2d-90f3-ac5a707d7192 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:30.157054 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1180/2353] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:17:30.157261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:30.180097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f3b55-671c-4c2e-8b07-8aca525666c6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:30.186605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:30.187719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:30.187993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:30.188391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] PATH_INFO: `/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd/users/388f279266084960967414017ebd9fcd/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:30.190523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f6ce9c64cd413986df3a0a2ac956cd, user_id=388f279266084960967414017ebd9fcd, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:30.196718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:30.211496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1174/2354] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:17:30 2022] PUT /identity/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd/users/388f279266084960967414017ebd9fcd/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:17:37.311420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:37.311963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:37.311963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:37.345231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] MFA Rules not processed for user `388f279266084960967414017ebd9fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:17:37.346924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:17:37.444744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1181/2355] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:17:37.455400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:37.461486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:37.559741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e71b-ecc5-4c7d-a8db-d6cec6623f83 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:37.565942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:37.567954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:37.567954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:37.567954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:37.646396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:37.648418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:37.670671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1175/2356] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:17:47.434621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:47.654494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=55 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:17:51.918286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:17:51.930802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ef2ca-eb71-42ce-a47f-a1046caba8c4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:17:51.936332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:17:51.936986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:17:51.937217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:17:51.937488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:17:51.995981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:17:51.997612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:17:52.007693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1182/2357] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:18:02.002756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=28 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:18:43.070371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:43.072074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:18:43.173161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409f0e-5b13-48de-854f-4ef9d7c87bc2 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:43.180854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:43.181487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:43.181799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:43.182196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:43.184173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:18:43.197256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:43.198979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:43.210779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1176/2358] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:18:46.700649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:46.716981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a57d5b-70af-425c-886c-66bd20b67f78 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:46.725246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:46.726376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:46.726769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:46.727283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:46.729828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:18:46.745513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:46.747381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:46.800815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:46.806224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1183/2359] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:18:46.914399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bf85b-418d-4d10-800d-ad35b65f5a96 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:46.919285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:46.919997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:46.920286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:46.920560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] PATH_INFO: `/v3/users/6d920e2427d145feb7ae253d5eda28b7` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:46.921762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d920e2427d145feb7ae253d5eda28b7)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:46.923775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:46.973359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d920e2427d145feb7ae253d5eda28b7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:46.974086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d920e2427d145feb7ae253d5eda28b7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:46.987130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d920e2427d145feb7ae253d5eda28b7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:46.997113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d920e2427d145feb7ae253d5eda28b7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:46.997958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d920e2427d145feb7ae253d5eda28b7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.011913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1177/2360] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:18:46 2022] DELETE /identity/v3/users/6d920e2427d145feb7ae253d5eda28b7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:18:47.233091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:47.248135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e73411-4a9e-430e-abd5-e48054289158 service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:47.326714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:47.327404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:47.327678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:47.328016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] PATH_INFO: `/v3/projects/2006ceb6c9d641019ec86aea7e4b676f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:47.329310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2006ceb6c9d641019ec86aea7e4b676f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:47.331258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:47.394644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2006ceb6c9d641019ec86aea7e4b676f', 'request_id': 'req-6af41ac5-f939-443c-96b6-f7939d86ee75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.402708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2006ceb6c9d641019ec86aea7e4b676f', 'request_id': 'req-6af41ac5-f939-443c-96b6-f7939d86ee75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.412419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2006ceb6c9d641019ec86aea7e4b676f', 'request_id': 'req-6af41ac5-f939-443c-96b6-f7939d86ee75'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.427989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] The token cache is being invalidate because project 2006ceb6c9d641019ec86aea7e4b676f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:18:47.428370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.428943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.430517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1184/2361] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:18:47 2022] DELETE /identity/v3/projects/2006ceb6c9d641019ec86aea7e4b676f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:18:47.434612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:47.436200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:18:47.451563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a58d-e0f8-4aba-8758-02ef2f7485b4 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:47.456635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:18:47.527375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:47.528136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:47.528411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:47.528684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] PATH_INFO: `/v3/users/388f279266084960967414017ebd9fcd` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:47.529929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388f279266084960967414017ebd9fcd)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:47.531897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:47.587300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388f279266084960967414017ebd9fcd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.587897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388f279266084960967414017ebd9fcd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.603643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388f279266084960967414017ebd9fcd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.611303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388f279266084960967414017ebd9fcd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.612000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388f279266084960967414017ebd9fcd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:47.624605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1178/2362] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:18:47 2022] DELETE /identity/v3/users/388f279266084960967414017ebd9fcd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:18:47.847159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:18:47.848776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:18:47.862435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af41ac5-f939-443c-96b6-f7939d86ee75 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:18:47.926596 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:18:47.927297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:18:47.927579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:18:47.927851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] PATH_INFO: `/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:18:47.929003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f6ce9c64cd413986df3a0a2ac956cd)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:18:47.934920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:18:47.959333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:18:47.995634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f6ce9c64cd413986df3a0a2ac956cd', 'request_id': 'req-350672c5-b812-4baa-8b77-40a8aab2dcc0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:48.001543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f6ce9c64cd413986df3a0a2ac956cd', 'request_id': 'req-350672c5-b812-4baa-8b77-40a8aab2dcc0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:48.010935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f6ce9c64cd413986df3a0a2ac956cd', 'request_id': 'req-350672c5-b812-4baa-8b77-40a8aab2dcc0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:48.024908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] The token cache is being invalidate because project f0f6ce9c64cd413986df3a0a2ac956cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:18:48.025197 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:48.025875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:18:48.027829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1185/2363] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:18:47 2022] DELETE /identity/v3/projects/f0f6ce9c64cd413986df3a0a2ac956cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:18:57.621225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:18:58.019942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:19:32.992833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:19:32.994289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:19:33.141086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bbefe-ff8c-4f1a-a2b0-326e8dbc0b06 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:19:33.201306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:19:33.201946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:19:33.202246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:19:33.202517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:19:33.204364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:19:33.215663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:19:33.217260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:19:33.227815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1179/2364] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:19:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:19:43.222975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:19:56.024956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:19:56.026372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:19:56.088561 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350672c5-b812-4baa-8b77-40a8aab2dcc0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:19:56.092183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:19:56.092783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:19:56.093059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:19:56.093329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:19:56.095059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:19:56.168099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:19:56.169544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:19:56.181050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1186/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:20:06.175700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=56 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:20:39.843155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:20:39.844487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:39.876252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c9c2a-e751-4c30-aba6-811fe6300de7 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:20:39.881875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:20:39.882481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:20:39.882760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:20:39.883034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:20:39.885602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:39.965296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:20:39.966941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:20:39.985744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1180/2366] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:20:46.687927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:20:46.689456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:46.700878 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb6b2-a14c-4988-81ad-0fe434878f35 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:20:46.704532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:20:46.705132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:20:46.705409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:20:46.705703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:20:46.707398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:46.721102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:20:46.722667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:20:46.733810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1187/2367] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:20:46.794822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:20:46.796221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:46.873279 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbddc9-97f8-4585-9e93-507f4871bca8 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:20:46.876992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:20:46.877594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:20:46.877902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:20:46.878175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:20:46.879891 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:46.935211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:20:46.955344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:20:46.956770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:20:46.966527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1181/2368] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:20:47.053326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:20:47.062680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1e488-09c3-455b-8c5b-0fccb071461f service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:20:47.066356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:20:47.067012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:20:47.067291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:20:47.067563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:20:47.069279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:47.151099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:20:47.152796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:20:47.171124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1188/2369] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:20:47.217887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:20:47.228733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b6e-90d5-43f9-8c1e-9242091d6b41 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:20:47.232600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:20:47.233194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:20:47.233475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:20:47.233782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:20:47.235531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:20:47.317995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:20:47.319454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cbf7f-ffe5-4c4c-8015-97ec12b5d6e9 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:20:47.339922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1182/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:20:57.168370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=47 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:20:57.335445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=51 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:21:37.115071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:21:37.134811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54c40-7072-47fa-af75-e7322860b1de service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:21:37.138620 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:21:37.139268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:21:37.139496 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:21:37.139763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:21:37.141457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:21:37.163418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:21:37.164833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:21:37.173295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1189/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:21:47.169877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=15 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:22:45.475722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:45.476253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:45.476253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:45.505573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] MFA Rules not processed for user `ccbb0b2070694235bf87a7b9a0c84dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:22:45.506845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:45.513854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1183/2372] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:22:45.826262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:22:45.834918 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0221b7-f7b8-43e8-81bf-e22e76b76b9a service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:22:45.838540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:22:45.839344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:45.839626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:45.839888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:45.841606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:45.851274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:22:45.852681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:22:45.860711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1190/2373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:22:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:22:46.192586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:22:46.194320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:46.210113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c50f3e-5ca2-4603-9e9b-264d14d9d984 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:22:46.215947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:22:46.216495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:46.216754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:46.217019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:46.218778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:46.231022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:22:46.232767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:22:46.243145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1184/2374] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:22:49.418355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:22:49.435678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2e1c3-8112-4392-8b58-c2e1a4d4795d service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:22:49.442011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:22:49.442867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:49.443154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:49.443429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:49.445223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:49.458755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:22:49.460616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:22:49.471814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1191/2375] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:22:54.822133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:22:54.836771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071bab1-5c6d-4679-9d70-2914d568d8a1 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:22:54.843036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:22:54.843639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:54.843912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:54.844184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:54.845930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:54.859785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:22:54.861463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:22:54.874103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1185/2376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:22:59.062313 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:22:59.063857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:59.078048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d90a3-dd73-41ff-b2d9-ad9f606cec29 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:22:59.083719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:22:59.084450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:22:59.084788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:22:59.085120 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:22:59.086939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:22:59.099549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:22:59.101169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:22:59.112703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1192/2377] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:23:03.330660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:23:03.332104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:23:03.343927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97372fb-c935-43f3-a0ef-a3b1b16072c1 service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:23:03.347543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:23:03.348133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:23:03.348411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:23:03.348685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:23:03.350479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:23:03.431901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:23:03.433973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:23:03.451997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1186/2378] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:23:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:23:09.108697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=18 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:23:13.449447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=59 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:24:38.346422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:24:38.348188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:24:38.362418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c2a9-2ad5-41ba-8ff9-79344bb0882f service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:24:38.367572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:24:38.368174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:24:38.368455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:24:38.368727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:24:38.377792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:24:38.380177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321291f-4c7f-47f0-8863-9de87362dfdd service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:24:38.390246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1193/2379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:24:48.385940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:25:04.410435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:25:04.422264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9216e-1fd2-453a-a5e2-6e82629add36 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:25:04.426237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:25:04.426950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:25:04.428111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:25:04.428976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:25:04.440549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:25:04.442039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:25:04.456001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1187/2380] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:25:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:25:11.304488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:25:11.305014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:25:11.305014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:25:11.324164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] MFA Rules not processed for user `e05967546ef245d2b88874f6cf3f84fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:25:11.325363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:25:11.332303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1194/2381] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:25:14.450731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:25:21.319187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:26:41.116585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:26:41.131621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f855984-6b91-4c2a-9ca9-adeafb0b7ed1 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:26:41.137438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:26:41.138077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:26:41.138356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:26:41.138627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:26:41.147671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:26:41.149278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:26:41.158551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1188/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:26:43.092844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:26:43.102707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04d938-e90f-4a69-914c-5ad95fa7a61b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:26:43.106929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:26:43.107529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:26:43.107808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:26:43.108079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:26:43.115374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:26:43.116802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:26:43.124853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1195/2383] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:26:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:26:47.472828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:26:47.482181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e4de-636f-4005-ad01-0372fe2c499b service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:26:47.485771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:26:47.486373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:26:47.486638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:26:47.486967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:26:47.496268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:26:47.497775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:26:47.526856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1189/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:26:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:26:53.121269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:26:57.524189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=19 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:27:04.596825 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:27:04.606890 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eea2bf-54ff-41a9-97c5-3a8687f68617 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:27:04.610556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:27:04.611152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:27:04.611432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:27:04.611705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:27:04.620435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:27:04.621902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:27:04.632519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1196/2385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:27:04.677521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:27:04.686593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3de954-0d0a-4c97-b754-ee61d92dd061 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:27:04.690356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:27:04.690949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:27:04.691231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:27:04.691502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:27:04.700471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:27:04.702028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:27:04.713057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1190/2386] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:27:14.627275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:27:14.707464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:28:47.636854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:28:47.646586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e6e8-548e-4189-8f63-56839dce36da service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:28:47.650520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:28:47.651142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:28:47.651418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:28:47.651689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:28:47.662290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:28:47.663793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:28:47.673808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1197/2387] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:28:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:28:57.669226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:01.820256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:01.836061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2be51-ed5d-43c4-aaac-64ab90c3cc58 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:01.839934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:01.840547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:01.840829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:01.841101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:01.852882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:01.854293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:01.867301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1191/2388] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:01.905945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:01.908303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:02.009191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae0b9-7000-4011-ae4a-a0b7d752940e service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:02.014402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:02.014402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:02.014708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:02.015025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:02.017115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] Could not find project: 2006ceb6c9d641019ec86aea7e4b676f.: keystone.exception.ProjectNotFound: Could not find project: 2006ceb6c9d641019ec86aea7e4b676f. Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 06 12:29:02.042336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 12:29:02.043753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 12:29:02.044961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application with Lock( Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._enter() Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.creator() Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application created_value = creator( Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 12:29:02.046769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 12:29:02.049554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2006ceb6c9d641019ec86aea7e4b676f. Sep 06 12:29:02.049554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: ERROR keystone.server.flask.application Sep 06 12:29:02.049554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1198/2389] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:01 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 139 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 06 12:29:03.409427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:03.426935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3323a-d348-4ca5-ad0d-18a923c905ee service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:03.436545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:03.437194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:03.437469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:03.437768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:03.440024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:03.556217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:03.557726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:03.593956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1192/2390] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:06.425085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:06.446749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be661b-4ea7-4000-90db-072de9cb0aa6 service glance] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:06.454435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:06.464694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:06.468076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:06.468649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:06.469994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:06.473444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9186-f60f-4589-8b1d-b328a28e91f9 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:06.480687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:06.481483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:06.481890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:06.482253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:06.482400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:06.484010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:06.496649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1199/2391] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:06.504683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:06.506530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:06.521856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1193/2392] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:07.560557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:07.570091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b8012-8d7f-4fa9-826e-fe7a159c581a service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:07.574017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:07.574674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:07.574952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:07.575223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:07.576996 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:07.704625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:07.705115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:07.730649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1200/2393] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:14.512696 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:14.523268 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dcc3a-b876-4c0b-b4b5-0f9410af1adf service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:14.527739 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:14.528344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:14.528620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:14.528892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:14.531982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:14.548348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:14.550713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:14.566061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1194/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:17.725201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=82 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:17.991215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.004238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f739c95-4c98-46cc-9e20-8b8a3eb971a2 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.018296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.019968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.020411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.022330 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.024476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:18.037943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.039848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.050977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1201/2395] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:18.073216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.075554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:18.101840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0d902-ade1-49bd-88a7-169ea94e7d9e service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.111254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.111903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.112768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.113056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] PATH_INFO: `/v3/users/41bc35497fe94ababe0c824a429168d3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.114263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bc35497fe94ababe0c824a429168d3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.116262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.181733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bc35497fe94ababe0c824a429168d3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.182524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bc35497fe94ababe0c824a429168d3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.196361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bc35497fe94ababe0c824a429168d3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.204927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bc35497fe94ababe0c824a429168d3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.205538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-974800d4-7a9d-4000-9ba2-9b491de9d809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bc35497fe94ababe0c824a429168d3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.221326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1195/2396] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:18 2022] DELETE /identity/v3/users/41bc35497fe94ababe0c824a429168d3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:18.415436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.428864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff7158-293a-4f6f-9186-6d37168165ef service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.491789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.492433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.492715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.492988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] PATH_INFO: `/v3/projects/ef6b481491244111a479eb91e3594ae4` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.494166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6b481491244111a479eb91e3594ae4)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.496089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.560512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6b481491244111a479eb91e3594ae4', 'request_id': 'req-e144f506-a2af-4a02-9c41-996db1662d12'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.566632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6b481491244111a479eb91e3594ae4', 'request_id': 'req-e144f506-a2af-4a02-9c41-996db1662d12'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.575086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6b481491244111a479eb91e3594ae4', 'request_id': 'req-e144f506-a2af-4a02-9c41-996db1662d12'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.587332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] The token cache is being invalidate because project ef6b481491244111a479eb91e3594ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:18.587736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.588372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e144f506-a2af-4a02-9c41-996db1662d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:18.589876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1202/2397] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:18 2022] DELETE /identity/v3/projects/ef6b481491244111a479eb91e3594ae4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:18.595600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.595939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.596222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.601758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:18.660624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:18.662466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:18.722102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1196/2398] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:18.730364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.730853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.730962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.751178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:18.752657 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:18.761420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1203/2399] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:18.765864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.776259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9dab2-5dd2-4ead-935a-8909ffcb5398 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.781196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.781841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.782130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.782403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.783166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:18.783474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.785088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.797532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1197/2400] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:18.801826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.811804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ba70d-d68c-4bd4-88fb-80e7b15b72c0 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.816458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.817050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.817323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.817588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.818398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.820272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.857136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1204/2401] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:18 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:18.861255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.871615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06687ac6-ab33-4c37-9a28-d2e4fc08320a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.930501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.931105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.931387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.931661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.932298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.934262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.956214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1198/2402] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:18 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:18.961028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:18.971525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7a295-57da-4a8d-80e7-d0bc414c8bc8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:18.976297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:18.976897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:18.977177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:18.977449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:18.978204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:18.980278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:18.990839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1205/2403] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:18.994833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:19.005009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40be52e-726c-4764-bee1-c33f14476075 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:19.016301 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:19.016919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:19.017195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:19.017460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:19.018139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:19.019749 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456dfd9-9b86-4211-a370-a9d6a9902afd admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:19.031252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1199/2404] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:19.035459 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:19.046220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db30221-b382-48be-9ec9-fa774d734a40 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:19.050964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:19.051616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:19.051897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:19.052655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] PATH_INFO: `/v3/projects/13f800639e444fd09c2f9a23d4db580e/users/b970d96d88a64f41b749c4c68619f20d/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:19.067468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f800639e444fd09c2f9a23d4db580e, user_id=b970d96d88a64f41b749c4c68619f20d, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:19.070961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:19.084631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1206/2405] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:19 2022] PUT /identity/v3/projects/13f800639e444fd09c2f9a23d4db580e/users/b970d96d88a64f41b749c4c68619f20d/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:19.094637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:19.095143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:19.095248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:19.123775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] MFA Rules not processed for user `b970d96d88a64f41b749c4c68619f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:19.124994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:19.152914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:19.194378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1200/2406] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:19.206452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:19.209304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:19.214602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:19.303814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdc0b-da6c-4809-a1f6-e119fe9fc633 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:19.309491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:19.310052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:19.310329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:19.310595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:19.319506 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:19.321102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c5b90-3166-4a7a-8b01-11ac9d76a543 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:19.331193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1207/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:22.120911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:22.183936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b86bb-d6e8-4913-aa14-86fa0ea4a06a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:22.188628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:22.189252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.189527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.189829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] PATH_INFO: `/v3/users/b970d96d88a64f41b749c4c68619f20d` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.190949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b970d96d88a64f41b749c4c68619f20d)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:22.192912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:22.240495 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b970d96d88a64f41b749c4c68619f20d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.242458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b970d96d88a64f41b749c4c68619f20d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.255040 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b970d96d88a64f41b749c4c68619f20d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.262447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b970d96d88a64f41b749c4c68619f20d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.263121 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b970d96d88a64f41b749c4c68619f20d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.277041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1201/2408] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:22 2022] DELETE /identity/v3/users/b970d96d88a64f41b749c4c68619f20d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:22.291505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.291982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.292074 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.312867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:22.316524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:22.378367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1208/2409] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:22.388254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:22.389703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:22.444544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:22.450727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2cd24-abc0-4317-bfda-7cf95a3e8605 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:22.455571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:22.456173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.456454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.456728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.466305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:22.467774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d14a-a7f6-47bf-a52b-c798a5c71f2f service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:22.477900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1202/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:22.754128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:22.755676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:22.773438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb7d06-102f-4d9d-a510-4fec7c9149fe None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:22.778589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:22.779337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.779599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.779868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] PATH_INFO: `/v3/projects/13f800639e444fd09c2f9a23d4db580e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.781010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f800639e444fd09c2f9a23d4db580e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:22.782954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:22.840485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f800639e444fd09c2f9a23d4db580e', 'request_id': 'req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.846087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f800639e444fd09c2f9a23d4db580e', 'request_id': 'req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.853789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f800639e444fd09c2f9a23d4db580e', 'request_id': 'req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.865063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] The token cache is being invalidate because project 13f800639e444fd09c2f9a23d4db580e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:22.865396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.866557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-98e0cd84-96f1-44ec-9454-15c2cf4cf59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:22.868262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1209/2411] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:22 2022] DELETE /identity/v3/projects/13f800639e444fd09c2f9a23d4db580e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:22.873137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.873418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.873716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.893713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:22.894914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:22.962492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1203/2412] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:22.971326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:22.971617 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:22.971945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:22.980425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:22.995294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:22.996554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:23.004735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1210/2413] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:23.009309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.019300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415920-c44c-464b-9811-7fa2351d914c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.024113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.024712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.024989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.025261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.025938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:23.026248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.028115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.040448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1204/2414] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:23.044814 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.055085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155b919-ba30-435b-a59f-5e37613e1314 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.059795 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.060391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.060665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.060931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.061556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.063471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.092504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1211/2415] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:23.096727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.107504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff140-a5a1-4af9-83e9-01afe58a23a8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.168226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.168834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.169115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.169449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.170131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.172061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.196602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1205/2416] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:23.200863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.211786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f437ed0-9ba7-408a-bbf3-9cff8d8322d5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.216498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.217099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.217376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.217672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.218324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.219916 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.230954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1212/2417] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:23.234797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.245708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1b4e-822e-40f4-bf45-7356c2360e35 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.250683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.251308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.251582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.251848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.252780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.254116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f082f-b207-412f-9a04-d2936179a571 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.265364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1206/2418] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:23.269583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.280782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae1044-454e-4380-aa46-96c6088981f3 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.285421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.286096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.286371 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.286633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] PATH_INFO: `/v3/projects/e48ee392697648e19b2e0162f153e60b/users/29ee38f72fce4081b44ffabf7a0fa451/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.301722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48ee392697648e19b2e0162f153e60b, user_id=29ee38f72fce4081b44ffabf7a0fa451, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.304395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.317172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1213/2419] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:23 2022] PUT /identity/v3/projects/e48ee392697648e19b2e0162f153e60b/users/29ee38f72fce4081b44ffabf7a0fa451/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:23.327444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.327757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.328031 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.358063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] MFA Rules not processed for user `29ee38f72fce4081b44ffabf7a0fa451`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:23.359190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:23.404880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:23.428029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1207/2420] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:23.433344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.497447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cc53-8a97-478b-8b27-6ff216f4f59b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.502224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.502841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.503113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.503382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.504025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.505944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.533664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1214/2421] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:23.537908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.548814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4817-4ea8-46b1-8598-2aa777145bcb None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.606195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.606799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.607085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.607359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.608006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.609995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.631306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1208/2422] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:23.635830 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.646376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d198a-c36b-4b5f-89dc-4c903a002c97 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.652682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.653332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.653602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.653914 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.654567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.656207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.668445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1215/2423] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:23.672652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.683009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e23923-f17b-4063-87a7-8fa0e28bd387 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.688059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.688712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.688994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.689264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] PATH_INFO: `/v3/projects/48d3243511d14aaaa1ddfb61d4393699/users/12b0677c4e0049a0ab22f7a9af0f3020/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.704549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d3243511d14aaaa1ddfb61d4393699, user_id=12b0677c4e0049a0ab22f7a9af0f3020, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.707265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.726151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1209/2424] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:23 2022] PUT /identity/v3/projects/48d3243511d14aaaa1ddfb61d4393699/users/12b0677c4e0049a0ab22f7a9af0f3020/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:23.735386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.746293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5da8-ed0a-4c40-a956-2b9c8a011068 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.802727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.803362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.803642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.803909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.804769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.806409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c3004-9e11-4da8-80e8-1bb4b9159af6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.816715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1216/2425] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:23.821703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.832243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c08fe-b17a-48f7-acf8-30cea7f439aa admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:23.837734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:23.838386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.838661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.839007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] PATH_INFO: `/v3/projects/48d3243511d14aaaa1ddfb61d4393699/users/12b0677c4e0049a0ab22f7a9af0f3020/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.840848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d3243511d14aaaa1ddfb61d4393699, user_id=12b0677c4e0049a0ab22f7a9af0f3020, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:23.843612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:23.854422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1210/2426] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:23 2022] PUT /identity/v3/projects/48d3243511d14aaaa1ddfb61d4393699/users/12b0677c4e0049a0ab22f7a9af0f3020/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:23.865894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:23.866440 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:23.866978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:23.890954 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:23.902919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] MFA Rules not processed for user `12b0677c4e0049a0ab22f7a9af0f3020`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:23.904137 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:23.975957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1217/2427] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:23.987767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:23.989790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:24.056820 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609abb0-ef21-41ed-a1fa-ab65d0ec3191 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:24.060935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:24.061132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:24.061396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:24.061758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:24.072232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:24.073714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:24.084116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1211/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:24.164901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:24.175085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e08c10-0892-474b-904b-353bd9a07536 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:24.179631 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:24.180066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:24.180342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:24.180607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:24.245479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:24.246973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2500d4c-641d-45a2-a2e9-05db2f9f303b service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:24.257411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1218/2429] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:34.079465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:34.252366 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=58 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:36.854657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:36.955407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069722e2-a56b-4c4a-acca-0c05db128d71 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:36.960396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:36.961041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:36.961356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:36.961598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] PATH_INFO: `/v3/users/29ee38f72fce4081b44ffabf7a0fa451` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:36.962877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ee38f72fce4081b44ffabf7a0fa451)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:36.965205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:37.018642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ee38f72fce4081b44ffabf7a0fa451'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.019341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ee38f72fce4081b44ffabf7a0fa451'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.032260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ee38f72fce4081b44ffabf7a0fa451'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.041606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ee38f72fce4081b44ffabf7a0fa451'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.042256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ee38f72fce4081b44ffabf7a0fa451'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.055976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1212/2430] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:36 2022] DELETE /identity/v3/users/29ee38f72fce4081b44ffabf7a0fa451 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:37.060919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:37.061450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:37.061548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:37.092729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:37.093881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:37.164362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1219/2431] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:37.174117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:37.175646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:37.263124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:37.280045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa491e-4dc5-427e-894e-fbcfaf33afa8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:37.287380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:37.288571 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:37.289050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:37.289463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:37.300048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:37.301842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:37.318326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1213/2432] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:37.679660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:37.681484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:37.699477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41150d6-e35b-4caa-957a-2399cb56ae45 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:37.710064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:37.710930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:37.711425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:37.728372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] PATH_INFO: `/v3/projects/e48ee392697648e19b2e0162f153e60b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:37.739378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48ee392697648e19b2e0162f153e60b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:37.762405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:37.867276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48ee392697648e19b2e0162f153e60b', 'request_id': 'req-4b44521e-3eec-4992-a74c-b119143c628b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.876165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48ee392697648e19b2e0162f153e60b', 'request_id': 'req-4b44521e-3eec-4992-a74c-b119143c628b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.889831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48ee392697648e19b2e0162f153e60b', 'request_id': 'req-4b44521e-3eec-4992-a74c-b119143c628b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.904601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] The token cache is being invalidate because project e48ee392697648e19b2e0162f153e60b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:37.904813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.905535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:37.910188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1220/2433] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:37 2022] DELETE /identity/v3/projects/e48ee392697648e19b2e0162f153e60b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:37.911644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:37.924139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da591-6bec-46c1-9a93-0cc1760649ee service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:38.034296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:38.035597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.035881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.036158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] PATH_INFO: `/v3/users/12b0677c4e0049a0ab22f7a9af0f3020` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.037908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b0677c4e0049a0ab22f7a9af0f3020)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:38.039926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:38.100211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b0677c4e0049a0ab22f7a9af0f3020'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.100897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b0677c4e0049a0ab22f7a9af0f3020'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.113308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b0677c4e0049a0ab22f7a9af0f3020'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.133901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b0677c4e0049a0ab22f7a9af0f3020'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.134642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5ec4def1-b57c-4176-b2a5-d3d9b450551b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b0677c4e0049a0ab22f7a9af0f3020'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.149567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1214/2434] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:37 2022] DELETE /identity/v3/users/12b0677c4e0049a0ab22f7a9af0f3020 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:38.531706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:38.533335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:38.547743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44521e-3eec-4992-a74c-b119143c628b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:38.610958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:38.611597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.611883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.612156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] PATH_INFO: `/v3/projects/48d3243511d14aaaa1ddfb61d4393699` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.613286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d3243511d14aaaa1ddfb61d4393699)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:38.615253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:38.632961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:38.668888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d3243511d14aaaa1ddfb61d4393699', 'request_id': 'req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.674867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d3243511d14aaaa1ddfb61d4393699', 'request_id': 'req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.684198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d3243511d14aaaa1ddfb61d4393699', 'request_id': 'req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.697205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] The token cache is being invalidate because project 48d3243511d14aaaa1ddfb61d4393699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:38.697494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.698171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-59832e29-b6a0-40a3-9837-4dbf4a6ba4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:38.699686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1221/2435] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:38 2022] DELETE /identity/v3/projects/48d3243511d14aaaa1ddfb61d4393699 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:38.704540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.704822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.705093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.726542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:38.727757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:38.794627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1215/2436] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:38.802765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.802993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.805828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.828899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:38.830122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:38.843166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1222/2437] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:38.844283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:38.866479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6bc2f-af82-48d3-98f2-ed7336e2b16d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:38.872222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:38.872842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.873193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.873433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.874441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:38.874744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:38.876453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:38.878231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:38.892164 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1216/2438] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:38.896279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:38.906580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1313-abe6-44e2-825c-e4e53e039813 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:38.911430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:38.912036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:38.912342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:38.912607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:38.913242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:38.915174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:38.944761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1223/2439] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:38 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:38.948758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:38.960285 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463ca02-0157-4763-970a-27568563a413 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.033499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.034265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.034659 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.035028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.035761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.037931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.064994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1217/2440] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:38 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:39.070305 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.081009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5d2d-fd53-42f3-aa68-55dffea1627c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.086069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.086676 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.086955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.087227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.087945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.089598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.100348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1224/2441] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:39.104464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.115815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc615057-7295-4cec-8bb5-e8a66cc5c868 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.120714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.121321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.121598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.121904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.122550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.124156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d640b-08a1-45e5-a316-66e10d85a6e5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.136369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1218/2442] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:39.140584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.151439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525af9b-3cd1-4b41-97bd-cda48ffa7849 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.156588 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.157327 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.157805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.158146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] PATH_INFO: `/v3/projects/c15c76db81a249a6806b4fe053c2bc3f/users/24a80984649749cf8b05795b392cbd31/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.176493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15c76db81a249a6806b4fe053c2bc3f, user_id=24a80984649749cf8b05795b392cbd31, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.180405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.195221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1225/2443] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:39 2022] PUT /identity/v3/projects/c15c76db81a249a6806b4fe053c2bc3f/users/24a80984649749cf8b05795b392cbd31/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:39.205458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.205786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.206072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.235039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] MFA Rules not processed for user `24a80984649749cf8b05795b392cbd31`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:39.236299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:39.313507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1219/2444] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:39.318005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.389398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d78db-9124-4b39-9ae0-e6e6eb9e5789 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.394968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.395543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.395928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.396189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.396999 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.398961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.433430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1226/2445] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:39 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:39.437116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.451508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea5e1-78c9-4f36-b50f-44316a4d8dfe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.554671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.555124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.555410 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.555961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.556639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.559957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.578391 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:39.602724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1220/2446] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:39 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:39.604584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.620608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab187b-cb8d-4a2a-a2ee-953f22a89323 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.637171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.638210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.638574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.638836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.639558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.641202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.654399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1227/2447] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:39.659741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.671597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590da21-b1f4-4ca6-b3d0-91ff477d3b1f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.676407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.677073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.677352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.677780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] PATH_INFO: `/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9/users/095a0f6fc67643eba2a8465be60e8491/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.693042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb14a3aacfda48bfab6ba19e363e2bf9, user_id=095a0f6fc67643eba2a8465be60e8491, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.696295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.711026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1221/2448] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:39 2022] PUT /identity/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9/users/095a0f6fc67643eba2a8465be60e8491/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:39.715271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.725769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ae3ee-a6b6-4c62-a3f8-221c043a8639 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.737589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:39.788413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.789065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.789345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.789618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.790438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.792085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debed7c-ce9b-47f4-b554-77248ef0ce12 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.802357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1228/2449] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:39.806959 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:39.829855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc5dd-0514-4013-8c0b-861633ed9174 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:39.835526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:39.836195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.836469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.836737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] PATH_INFO: `/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9/users/095a0f6fc67643eba2a8465be60e8491/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.838680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb14a3aacfda48bfab6ba19e363e2bf9, user_id=095a0f6fc67643eba2a8465be60e8491, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:39.841356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:39.852384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1222/2450] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:39 2022] PUT /identity/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9/users/095a0f6fc67643eba2a8465be60e8491/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:39.864085 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:39.864554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:39.864780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:39.899510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] MFA Rules not processed for user `095a0f6fc67643eba2a8465be60e8491`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:39.900783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:39.986983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1229/2451] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:40.001094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:40.002973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:40.073225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf2c0-af98-4dde-9845-9cd25f7c4aae admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:40.077657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:40.078358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:40.078927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:40.079453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:40.090955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:40.096361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:40.111197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1223/2452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:29:40.478800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:40.487942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8caf7f-8a54-4c3f-8041-6c852d039393 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:40.493940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a0f6fc67643eba2a8465be60e8491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:40.494555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:40.494836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:40.495375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:40.496058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:40.496527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:40.499009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:40.512175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1230/2453] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:40.516153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:40.526010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432e1da-92a3-4b5c-a011-ea495da74347 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:40.531068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a0f6fc67643eba2a8465be60e8491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:40.531684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:40.531963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:40.532239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:40.532915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:40.534872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:40.567417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1224/2454] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:40 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:40.568720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:40.579107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5985a-3b33-48b1-9bbd-9500420269f7 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:40.654866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a0f6fc67643eba2a8465be60e8491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:40.654866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:40.654866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:40.655917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:40.655917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:40.655917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:40.655917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:40.669392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1231/2455] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 12:29:40.731813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:40.744769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e0518-875e-4161-901d-59fe80c6d7ab tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:40.763911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a0f6fc67643eba2a8465be60e8491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:40.764589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:40.764877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:40.766515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] PATH_INFO: `/v3/projects/cb05ef8531c24c8bbca64fc39d284845` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:40.772328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cb05ef8531c24c8bbca64fc39d284845)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:40.775047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:40.874037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb05ef8531c24c8bbca64fc39d284845', 'request_id': 'req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:40.887200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb05ef8531c24c8bbca64fc39d284845', 'request_id': 'req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:40.897599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb05ef8531c24c8bbca64fc39d284845', 'request_id': 'req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:40.921035 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] The token cache is being invalidate because project cb05ef8531c24c8bbca64fc39d284845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:40.921750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:40.923723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:40.929783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1225/2456] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:40 2022] DELETE /identity/v3/projects/cb05ef8531c24c8bbca64fc39d284845 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:41.021014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:41.022975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.053828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:41.154164 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11819a3c-6038-44dd-b5e0-8279bf959e2a tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:41.210193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:41.211965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.243483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:41.244507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:41.245212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:41.245587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:41.245919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:41.248083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.269931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:41.272187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c251-1f47-4f0a-8717-b2040a38c123 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:41.284766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1232/2457] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:41.311180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e276-8bc6-4ba8-aabc-c2861e982ef5 tempest-VolumeQuotaClassesTest-871715213 tempest-VolumeQuotaClassesTest-871715213-project-admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:41.316070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:41.317220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:41.317220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:41.317546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] PATH_INFO: `/v3/users/24a80984649749cf8b05795b392cbd31` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:41.319860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a80984649749cf8b05795b392cbd31)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:41.321883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:41.373851 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a80984649749cf8b05795b392cbd31'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:41.374535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a80984649749cf8b05795b392cbd31'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:41.387292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a80984649749cf8b05795b392cbd31'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:41.395816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a80984649749cf8b05795b392cbd31'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:41.397039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a80984649749cf8b05795b392cbd31'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:41.410814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1226/2458] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:41 2022] DELETE /identity/v3/users/24a80984649749cf8b05795b392cbd31 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:41.415468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:41.415836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:41.416113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:41.435623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:41.436848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.499929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1233/2459] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:41.509946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:41.511566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.577665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7bc-2f29-4fd3-8d09-3d87f22ea774 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:41.581772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:41.582511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:41.582831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:41.583071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:41.596493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:41.598318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:41.610940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1227/2460] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:41.949058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:41.950395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:41.964122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b93d3-702f-4054-bd7e-20d994c07e29 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:41.970189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:41.970818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:41.971092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:41.971365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] PATH_INFO: `/v3/projects/c15c76db81a249a6806b4fe053c2bc3f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:41.972557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15c76db81a249a6806b4fe053c2bc3f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:41.975102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:42.015292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:42.057193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15c76db81a249a6806b4fe053c2bc3f', 'request_id': 'req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.065098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15c76db81a249a6806b4fe053c2bc3f', 'request_id': 'req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.078888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15c76db81a249a6806b4fe053c2bc3f', 'request_id': 'req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.095885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] The token cache is being invalidate because project c15c76db81a249a6806b4fe053c2bc3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:42.096182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.096920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.101898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1234/2461] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:41 2022] DELETE /identity/v3/projects/c15c76db81a249a6806b4fe053c2bc3f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:42.103155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:42.131054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134347-8869-4ac2-8b21-31d7c37308f7 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:42.227881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:42.256376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:42.257704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:42.258134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:42.258547 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] PATH_INFO: `/v3/users/095a0f6fc67643eba2a8465be60e8491` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:42.260132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=095a0f6fc67643eba2a8465be60e8491)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:42.262392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:42.326237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095a0f6fc67643eba2a8465be60e8491'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.326957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095a0f6fc67643eba2a8465be60e8491'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.346715 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095a0f6fc67643eba2a8465be60e8491'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.356612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095a0f6fc67643eba2a8465be60e8491'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.357500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2cdfe10e-b7e1-4e01-b24c-e7e5db1eb0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095a0f6fc67643eba2a8465be60e8491'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.375003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1228/2462] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:42 2022] DELETE /identity/v3/users/095a0f6fc67643eba2a8465be60e8491 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:42.728436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:42.729990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:42.743489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2e0e9-5b51-4e6e-ade0-3f408a7c7819 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:42.805385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:42.805948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:42.806228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:42.806498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] PATH_INFO: `/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:42.807626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb14a3aacfda48bfab6ba19e363e2bf9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:42.809533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:42.857274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'request_id': 'req-5024fcff-92f5-4df2-995b-6faa5e1d315d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.863437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'request_id': 'req-5024fcff-92f5-4df2-995b-6faa5e1d315d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.871948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb14a3aacfda48bfab6ba19e363e2bf9', 'request_id': 'req-5024fcff-92f5-4df2-995b-6faa5e1d315d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.884283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] The token cache is being invalidate because project eb14a3aacfda48bfab6ba19e363e2bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:42.884567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.885217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-5024fcff-92f5-4df2-995b-6faa5e1d315d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:42.886771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1235/2463] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:42 2022] DELETE /identity/v3/projects/eb14a3aacfda48bfab6ba19e363e2bf9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:42.891829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:42.892132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:42.892393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:42.913563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:42.914871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:42.977264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1229/2464] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:42.985070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:42.985613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:42.985763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.007073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:43.008428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:43.016944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1236/2465] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:29:43.021356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.031657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d3962-8836-4b16-8943-7c5c9a3dbc54 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.036329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.036940 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.037248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.037564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.038288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:43.038594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.040215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.052082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1230/2466] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:43.057730 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.067850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71ed12-54fe-4f9b-b25a-b8e90e27451d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.074133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.074826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.075169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.075488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.076175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.078282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.106555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1237/2467] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:43.111033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.122010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c73f-aceb-43ed-ba91-7bcdd080e1f1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.168100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:43.183769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.184419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.184717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.184975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.185610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.187658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.223881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1231/2468] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:43.228669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.239597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1220-66f3-4233-95f5-faf831d2c340 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.246642 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.247286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.247565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.247836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.248487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.250209 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.261543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1238/2469] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:43.265748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.276429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9d58-62f3-42f9-98b5-4b1537c92f17 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.281932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.282488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.282776 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.283033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.283848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.285470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da1dd-1abf-4763-a7b1-99d0a40a09d4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.300227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1232/2470] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:43.301761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.310560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:43.315048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350c7b0-8a61-4802-a7c2-ce8bc736de6c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.321504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.322178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.322464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.322919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] PATH_INFO: `/v3/projects/7ac262867e604585bb8be843abe99f27/users/b781141dcac0476d837c48297ad9c534/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.338465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac262867e604585bb8be843abe99f27, user_id=b781141dcac0476d837c48297ad9c534, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.341144 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.355130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1239/2471] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:43 2022] PUT /identity/v3/projects/7ac262867e604585bb8be843abe99f27/users/b781141dcac0476d837c48297ad9c534/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:43.366803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.367107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.367379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.394700 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] MFA Rules not processed for user `b781141dcac0476d837c48297ad9c534`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:43.395943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:43.468522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1233/2472] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:43.472458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.517939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.519358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:43.539962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e506f-57f1-4fc2-bf2f-cf2f27044f63 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.544774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.545383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.545687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.545961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.546610 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.548582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.580188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1240/2473] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:43.581081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.585862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d5b5a-449b-4ee4-965d-d876ed83ccb3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.637780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.638397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.638675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.638956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.640790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:43.647724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b810b3-4eb2-43ca-b82e-c40f6fc21cbf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.652970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.653587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.653960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.654170 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.654371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.654802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.655826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.656781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.666931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1234/2474] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:29:43.681027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1241/2475] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:43.685099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.695060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f44bac-cc0e-4943-896a-55134eaa6b9b service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.699858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.700445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.700719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.701002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.701620 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.703277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.717499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1235/2476] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:43.719413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.732378 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ad44b-4d0a-406d-9b2f-40b9408c7fd7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.736657 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.737333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.737613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.737946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] PATH_INFO: `/v3/projects/3fb9247928d241b495b63c457e152ab0/users/18cbbe704a6e49dea2b1f792da82148f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.757923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb9247928d241b495b63c457e152ab0, user_id=18cbbe704a6e49dea2b1f792da82148f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.760877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.780403 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1242/2477] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:43 2022] PUT /identity/v3/projects/3fb9247928d241b495b63c457e152ab0/users/18cbbe704a6e49dea2b1f792da82148f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:43.784736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.792440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:43.804059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299cd8-8b37-46b4-a2ae-6c2c8df689d2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.892263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.892874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.893156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.893430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.894136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.895758 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ee33e-f7bd-4e89-9c59-7d05ab359348 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.907105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1236/2478] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:43.911485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:43.923129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91c0dd-2cf6-45ad-b4db-f4f344431493 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:43.931771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:43.932487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.932766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.933034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] PATH_INFO: `/v3/projects/3fb9247928d241b495b63c457e152ab0/users/18cbbe704a6e49dea2b1f792da82148f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:43.935100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb9247928d241b495b63c457e152ab0, user_id=18cbbe704a6e49dea2b1f792da82148f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:43.937800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:43.949535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1243/2479] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:43 2022] PUT /identity/v3/projects/3fb9247928d241b495b63c457e152ab0/users/18cbbe704a6e49dea2b1f792da82148f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:43.961486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b70f1-2081-480d-856b-0453be9a421c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:43.961810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b70f1-2081-480d-856b-0453be9a421c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:43.962091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b70f1-2081-480d-856b-0453be9a421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:44.000512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-450b70f1-2081-480d-856b-0453be9a421c None None] MFA Rules not processed for user `18cbbe704a6e49dea2b1f792da82148f`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:44.001805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-450b70f1-2081-480d-856b-0453be9a421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:44.075576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1237/2480] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:44.087715 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:44.089714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:44.112628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:44.155320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fd838-a931-4ed4-8b6f-75768207f3e0 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:44.159323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:44.159984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:44.160262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:44.160537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:44.171224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:44.173003 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05fb3b-3443-437c-8fe3-fb9dc5ad73d5 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:44.183322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1244/2481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:44.805250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b70f1-2081-480d-856b-0453be9a421c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:44.900219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b70f1-2081-480d-856b-0453be9a421c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:44.906095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:44.906843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:44.907372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:44.907744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] PATH_INFO: `/v3/users/b781141dcac0476d837c48297ad9c534` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:44.909224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b781141dcac0476d837c48297ad9c534)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:44.912945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:44.957299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:44.997325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b781141dcac0476d837c48297ad9c534'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:44.998349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b781141dcac0476d837c48297ad9c534'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.012840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b781141dcac0476d837c48297ad9c534'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.024144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b781141dcac0476d837c48297ad9c534'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.029420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b781141dcac0476d837c48297ad9c534'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.057351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1238/2482] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:44 2022] DELETE /identity/v3/users/b781141dcac0476d837c48297ad9c534 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:45.059299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:45.059587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:45.059926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:45.089866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:45.091117 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:45.164343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1245/2483] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:45.174662 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:45.176124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:45.236591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fabdd-d5d7-47c6-9fba-7d362007e085 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:45.240892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:45.241468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:45.241768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:45.242034 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:45.254324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:45.256352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:45.267128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1239/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:45.781282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:45.783425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:45.809534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9247be-9367-4453-b266-88bf3c911560 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:45.824412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:45.828329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:45.828743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:45.830470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] PATH_INFO: `/v3/projects/7ac262867e604585bb8be843abe99f27` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:45.831862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac262867e604585bb8be843abe99f27)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:45.834669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:45.900609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac262867e604585bb8be843abe99f27', 'request_id': 'req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.907784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac262867e604585bb8be843abe99f27', 'request_id': 'req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.918087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac262867e604585bb8be843abe99f27', 'request_id': 'req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.932639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] The token cache is being invalidate because project 7ac262867e604585bb8be843abe99f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:45.933099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.933919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:45.935569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1246/2485] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:45 2022] DELETE /identity/v3/projects/7ac262867e604585bb8be843abe99f27 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:45.939644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:45.950359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c8179-263a-479b-aa95-4546bd36f5d7 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:46.018603 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:46.019238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.019521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.019795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] PATH_INFO: `/v3/users/18cbbe704a6e49dea2b1f792da82148f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.020988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cbbe704a6e49dea2b1f792da82148f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:46.023658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:46.073115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cbbe704a6e49dea2b1f792da82148f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.073907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cbbe704a6e49dea2b1f792da82148f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.075358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:46.092408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cbbe704a6e49dea2b1f792da82148f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.105210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cbbe704a6e49dea2b1f792da82148f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.105981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cbbe704a6e49dea2b1f792da82148f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.119815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1240/2486] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:45 2022] DELETE /identity/v3/users/18cbbe704a6e49dea2b1f792da82148f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:46.301053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:46.303108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:46.430447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:46.442781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094ec61-6c4a-4db9-971d-4d3c3b8c0921 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:46.502195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:46.504141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:46.523682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:46.524323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.524601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.524968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.528720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:46.611471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:46.612928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa6d63-4b89-44e9-b860-900bb6507bbb service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:46.626447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1247/2487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:46.658101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0c880-c1fa-4241-a953-e61487ede92c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:46.664231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:46.664938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.665156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.665419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] PATH_INFO: `/v3/projects/3fb9247928d241b495b63c457e152ab0` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.666602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb9247928d241b495b63c457e152ab0)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:46.669093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:46.726788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb9247928d241b495b63c457e152ab0', 'request_id': 'req-432f1c46-ec96-4793-82de-9ec8e62bd1eb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.734717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb9247928d241b495b63c457e152ab0', 'request_id': 'req-432f1c46-ec96-4793-82de-9ec8e62bd1eb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.741025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb9247928d241b495b63c457e152ab0', 'request_id': 'req-432f1c46-ec96-4793-82de-9ec8e62bd1eb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.752352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] The token cache is being invalidate because project 3fb9247928d241b495b63c457e152ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:46.752641 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.754455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-432f1c46-ec96-4793-82de-9ec8e62bd1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:46.756848 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1241/2488] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:46 2022] DELETE /identity/v3/projects/3fb9247928d241b495b63c457e152ab0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:46.763955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.764359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.764760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.792226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:46.793452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:46.867118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1248/2489] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:29:46.875469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.880496 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.885305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.913808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:46.915098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:46.923288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1242/2490] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:46.930012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:46.940851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a03ada-3f8e-4e34-8427-9ae5860e1d31 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:46.946354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:46.946973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.947265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.947528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.948190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:46.948499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:46.950202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:46.963481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1249/2491] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:46.967855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:46.977935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98fcf-595d-4875-b11f-8b46592aa92b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:46.984085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:46.985068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:46.985501 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:46.985941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:46.989586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:46.992240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.035170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1243/2492] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:46 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:47.036391 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.048805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633fa0-000e-4723-bc94-eda643815ae8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.124910 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:47.165718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.174190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.174700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.175154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.176405 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.179111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.215823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1250/2493] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:47 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:47.219837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.239188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028da724-d617-488e-baf4-9b1c06166c17 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.252984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.253618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.253924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.254191 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.254837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.256536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.275047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1244/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:47.279286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.291683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471369e2-8fc7-4489-81eb-a93a4695b397 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.297166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.298296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.298685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.299100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.299855 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.301767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12708d-1ff1-4cb8-b625-36bc5a81d7ef admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.314906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1251/2495] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:47.319557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.334417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70204-4486-4879-b283-53ef6d99f5df admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.342800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.343901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.344282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.344660 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] PATH_INFO: `/v3/projects/26fe7ddc88334bc6a7912c5fa07b9516/users/71dcf98c074847158fabdb71a7594902/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.369556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:47.379290 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fe7ddc88334bc6a7912c5fa07b9516, user_id=71dcf98c074847158fabdb71a7594902, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.383490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.410596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1245/2496] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:47 2022] PUT /identity/v3/projects/26fe7ddc88334bc6a7912c5fa07b9516/users/71dcf98c074847158fabdb71a7594902/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:47.421764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.422396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.422678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.493345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] MFA Rules not processed for user `71dcf98c074847158fabdb71a7594902`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:47.495001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:47.599640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1252/2497] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:47.610915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.612852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:47.692750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:47.695390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:47.698088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb153a0-6f60-4f26-9629-bc148a67673b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.702618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.703406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.703687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.703992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.714367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.715834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.727248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1246/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:47.780470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca528f-8d1b-42e2-b436-5187e4b5e8a8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:47.786269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:47.787202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:47.787721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:47.788023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] PATH_INFO: `/v3/users/e05967546ef245d2b88874f6cf3f84fd` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:47.789204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05967546ef245d2b88874f6cf3f84fd)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:47.791184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:47.854368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05967546ef245d2b88874f6cf3f84fd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:47.857475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:47.868097 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05967546ef245d2b88874f6cf3f84fd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:47.891603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05967546ef245d2b88874f6cf3f84fd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:47.892425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05967546ef245d2b88874f6cf3f84fd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:47.907424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-134bb4d7-24ff-450a-9e98-ead7549b21ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05967546ef245d2b88874f6cf3f84fd'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:47.922006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1253/2499] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:47 2022] DELETE /identity/v3/users/e05967546ef245d2b88874f6cf3f84fd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:48.200295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.224032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d22cd6-7e80-4e3c-bdae-faf009457845 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.344476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.345406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.345709 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.346173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] PATH_INFO: `/v3/projects/eb4aab6e4d8f4582a2538ccd0004046c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.347381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4aab6e4d8f4582a2538ccd0004046c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.349961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.407366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4aab6e4d8f4582a2538ccd0004046c', 'request_id': 'req-5293b6e4-8625-41f3-a067-a5da65744f03'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:48.416047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4aab6e4d8f4582a2538ccd0004046c', 'request_id': 'req-5293b6e4-8625-41f3-a067-a5da65744f03'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:48.422523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4aab6e4d8f4582a2538ccd0004046c', 'request_id': 'req-5293b6e4-8625-41f3-a067-a5da65744f03'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:48.438545 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] The token cache is being invalidate because project eb4aab6e4d8f4582a2538ccd0004046c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:48.438744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:48.439384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5293b6e4-8625-41f3-a067-a5da65744f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:48.440892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1247/2500] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:48 2022] DELETE /identity/v3/projects/eb4aab6e4d8f4582a2538ccd0004046c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:48.446798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.447370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.447465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.469465 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:48.470792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:48.544467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1254/2501] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:48.553085 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.553583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.553915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.579585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:48.581382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:48.594989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1248/2502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:48.596453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.607444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79007f-029d-448e-b6ef-d934c649ea2d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.612492 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.613132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.613407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.613868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.614551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:48.614854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.616537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.632253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1255/2503] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:48.637977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.650206 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319054e-1044-4cd0-ba95-ec1c197eeeb2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.655979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.656922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.657358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.658393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.659295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.661499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.680797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:48.698728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1249/2504] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:48 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:48.701474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.713233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649ad6-2b97-49f1-92d6-843dc25e2452 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.814203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.814903 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.815192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.815467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.816122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.819450 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.845691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1256/2505] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:48 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:48.849157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.860754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea0a7-709a-4d2c-bb00-aac203e818ef admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.870851 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.871847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.872239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.872667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.873481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.875477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.892216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1250/2506] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:48.896627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.909199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c6cce-456e-42a2-8ba0-c053fd9a9a10 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.916029 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.916690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.916970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.918251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.918251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.920202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd46f3-a979-4a84-bce3-52efd99bd4dc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:48.938043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1257/2507] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:48.941142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:48.955805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46d0c-2129-45f1-b2a4-822d27993240 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:48.962730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:48.963553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:48.963863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:48.964139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] PATH_INFO: `/v3/projects/63975e75593d4ef587335840ead7339d/users/d93cc6b37ce145a29915e8b6b421e35f/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:48.984497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63975e75593d4ef587335840ead7339d, user_id=d93cc6b37ce145a29915e8b6b421e35f, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:48.987250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.006011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1251/2508] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:48 2022] PUT /identity/v3/projects/63975e75593d4ef587335840ead7339d/users/d93cc6b37ce145a29915e8b6b421e35f/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:49.014774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cebdaf-a0b4-40f2-9ac3-21e930102573 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.015065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cebdaf-a0b4-40f2-9ac3-21e930102573 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.015345 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cebdaf-a0b4-40f2-9ac3-21e930102573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.049042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-74cebdaf-a0b4-40f2-9ac3-21e930102573 None None] MFA Rules not processed for user `d93cc6b37ce145a29915e8b6b421e35f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:49.050436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-74cebdaf-a0b4-40f2-9ac3-21e930102573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:49.073311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:49.135153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1258/2509] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:49.139473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.230043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13dca-e886-4244-8431-72505d71034a None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.230358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13dca-e886-4244-8431-72505d71034a None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.230592 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13dca-e886-4244-8431-72505d71034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.232134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfb2e-fad5-4954-88c7-02f3e2acec67 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.237971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.238672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.239251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.239818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.241101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.243307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.266811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-31d13dca-e886-4244-8431-72505d71034a None None] MFA Rules not processed for user `71dcf98c074847158fabdb71a7594902`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:49.268348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-31d13dca-e886-4244-8431-72505d71034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:49.290626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1252/2510] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:49 2022] POST /identity/v3/projects => generated 405 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:49.291732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.357510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1259/2511] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:29:49.374703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c5b66-0f36-460b-9707-9f13885d31c1 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.381996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.385580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.386198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.386497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.387539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.389635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.418092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1253/2512] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:49 2022] POST /identity/v3/users => generated 309 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:49.419290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d13dca-e886-4244-8431-72505d71034a None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.433952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d13dca-e886-4244-8431-72505d71034a None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.439744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.440437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.440786 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.441069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.441970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.444679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.455798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1260/2513] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:49.461220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.471588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8fe6-29de-4f08-9668-d9fce65169d5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.476400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.477180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.477460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.477768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] PATH_INFO: `/v3/projects/84823bcee8834bd7b3f8c135a1db0693/users/0c4a56364c7b41e08971d8dfef1e02ab/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.493564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84823bcee8834bd7b3f8c135a1db0693, user_id=0c4a56364c7b41e08971d8dfef1e02ab, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.496405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.511987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1254/2514] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:49 2022] PUT /identity/v3/projects/84823bcee8834bd7b3f8c135a1db0693/users/0c4a56364c7b41e08971d8dfef1e02ab/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:49.516142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.527229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4979a-49cc-4211-abfc-dc5602c77ff3 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.591095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.591621 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.592962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.592962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.592962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.594423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ccfb-7d67-46a5-8f41-6c60f28d36c0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.607727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1261/2515] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:49.609398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.622020 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123cad7-9e2f-46a4-8650-655af1e11409 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.636089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.636865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.637217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.637565 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] PATH_INFO: `/v3/projects/84823bcee8834bd7b3f8c135a1db0693/users/0c4a56364c7b41e08971d8dfef1e02ab/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.639710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84823bcee8834bd7b3f8c135a1db0693, user_id=0c4a56364c7b41e08971d8dfef1e02ab, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.642671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.650462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:49.658139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1255/2516] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:49 2022] PUT /identity/v3/projects/84823bcee8834bd7b3f8c135a1db0693/users/0c4a56364c7b41e08971d8dfef1e02ab/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:49.670111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.670958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.671231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.707458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] MFA Rules not processed for user `0c4a56364c7b41e08971d8dfef1e02ab`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:49.708753 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:49.765673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:49.813868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1262/2517] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:49.825497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:49.827775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:49.892396 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aced-b91d-4529-87e0-8317dfabaa9b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:49.897690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:49.897867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:49.898045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:49.898287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:49.909325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:49.911006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:49.929722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1256/2518] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:50.078196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:50.079942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:50.109011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:50.228126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588fe0d-aeba-4ff5-92b0-defd40fafdac None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:50.237671 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:50.238359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:50.238639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:50.238913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:50.243150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:50.252870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963781-61c0-4b10-936b-e3e7066a8df4 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:50.258770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:50.260232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:50.260961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:50.261333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] PATH_INFO: `/v3/users/d93cc6b37ce145a29915e8b6b421e35f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:50.264890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93cc6b37ce145a29915e8b6b421e35f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:50.266992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:50.267685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:50.278946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee65c-078e-481d-bb98-c489ecca669a service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:50.298758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1263/2519] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:50.356796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93cc6b37ce145a29915e8b6b421e35f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:50.357818 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93cc6b37ce145a29915e8b6b421e35f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:50.376921 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93cc6b37ce145a29915e8b6b421e35f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:50.389052 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93cc6b37ce145a29915e8b6b421e35f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:50.391163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93cc6b37ce145a29915e8b6b421e35f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:50.414397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1257/2520] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:50 2022] DELETE /identity/v3/users/d93cc6b37ce145a29915e8b6b421e35f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:50.420196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:50.420512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:50.420819 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:50.463206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:50.465044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:50.587843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1264/2521] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:50.602110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:50.609500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:50.664978 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:50.736055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead91647-c704-4c24-8b99-fd1c98de5c61 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:50.743869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:50.745135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:50.745595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:50.746582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:50.759909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:50.761745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:50.774841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1258/2522] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:51.040630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.042389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.058168 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ed443-4840-4675-a4db-edfbf50899d6 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:51.066099 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:51.067714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.068622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.069401 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] PATH_INFO: `/v3/users/ccbb0b2070694235bf87a7b9a0c84dbe` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.071580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbb0b2070694235bf87a7b9a0c84dbe)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:51.074958 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:51.146984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbb0b2070694235bf87a7b9a0c84dbe'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.164411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbb0b2070694235bf87a7b9a0c84dbe'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.173390 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:51.187667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbb0b2070694235bf87a7b9a0c84dbe'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.188793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbb0b2070694235bf87a7b9a0c84dbe'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.202852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbb0b2070694235bf87a7b9a0c84dbe'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.212221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1265/2523] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:51 2022] DELETE /identity/v3/users/ccbb0b2070694235bf87a7b9a0c84dbe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:51.302787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.303625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.318882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ae62-4cae-4fbf-a354-d8f19a84f00c service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:51.406664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:51.407597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.407793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.408070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] PATH_INFO: `/v3/projects/63975e75593d4ef587335840ead7339d` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.409308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63975e75593d4ef587335840ead7339d)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:51.411330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:51.471918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63975e75593d4ef587335840ead7339d', 'request_id': 'req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.479868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.481171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63975e75593d4ef587335840ead7339d', 'request_id': 'req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.482148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.488231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63975e75593d4ef587335840ead7339d', 'request_id': 'req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.499866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d729-29e9-40a5-b644-aa236675965f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:51.506936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] The token cache is being invalidate because project 63975e75593d4ef587335840ead7339d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:51.507127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.515992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.517722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1259/2524] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:51 2022] DELETE /identity/v3/projects/63975e75593d4ef587335840ead7339d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:51.521941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.523931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.577758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:51.578437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.578820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.579004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] PATH_INFO: `/v3/projects/2be7be1a9bb1487aac2c32871398119e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.580203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be7be1a9bb1487aac2c32871398119e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:51.582201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:51.585930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:51.605962 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9b219-8cfa-4585-b1a3-860dbffb0dc7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:51.612149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:51.613460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.614000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.614398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] PATH_INFO: `/v3/users/0c4a56364c7b41e08971d8dfef1e02ab` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.616878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4a56364c7b41e08971d8dfef1e02ab)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:51.621246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:51.651458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be7be1a9bb1487aac2c32871398119e', 'request_id': 'req-e200dab9-0595-4bc8-bf2b-58fe170b13ce'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.660408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be7be1a9bb1487aac2c32871398119e', 'request_id': 'req-e200dab9-0595-4bc8-bf2b-58fe170b13ce'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.671190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be7be1a9bb1487aac2c32871398119e', 'request_id': 'req-e200dab9-0595-4bc8-bf2b-58fe170b13ce'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.687911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] The token cache is being invalidate because project 2be7be1a9bb1487aac2c32871398119e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:51.688198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.689265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e200dab9-0595-4bc8-bf2b-58fe170b13ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.690948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1266/2525] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:51 2022] DELETE /identity/v3/projects/2be7be1a9bb1487aac2c32871398119e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:51.692415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4a56364c7b41e08971d8dfef1e02ab'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.693226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4a56364c7b41e08971d8dfef1e02ab'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.696360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.696906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.697679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.708728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4a56364c7b41e08971d8dfef1e02ab'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.719272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4a56364c7b41e08971d8dfef1e02ab'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.721932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-8678e34c-f6fc-426a-90da-bdcdc72bf5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4a56364c7b41e08971d8dfef1e02ab'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:51.724762 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:51.726290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.739525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1260/2526] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:29:51 2022] DELETE /identity/v3/users/0c4a56364c7b41e08971d8dfef1e02ab => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:51.798564 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1267/2527] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:29:51.806403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.806691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.806968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.831521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:51.834117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:51.932124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1261/2528] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:51.932586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.944058 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b5594-d5c3-4aa9-8301-246ecd2ffac7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:51.950261 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:51.950992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:51.951396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:51.951747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:51.952638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:51.953020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:51.956043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:51.959896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:51.986854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:51.991696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1268/2529] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:51.997200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620f8d-ed8c-425b-b9e3-da112bd53b00 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.002900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.004517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.004884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.005630 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.006516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.008878 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.052260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1262/2530] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:51 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:52.056904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.068664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed086c80-6c4d-4d53-9753-272ed03afc6b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.121946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.123903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:52.165423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.168509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.168800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.169369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.170258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.174948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.214651 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:52.217032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1269/2531] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:52.222166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.236089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623c15-0057-4170-963b-b7aadd723d89 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.236342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded9823-6d97-4737-ba41-0f80bd230295 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.242695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.243552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.243979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.244226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.244572 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.245028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.245377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.246125 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.248591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.249956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] PATH_INFO: `/v3/projects/84823bcee8834bd7b3f8c135a1db0693` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.254095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] RBAC: Authorizing `identity:delete_project(project_id=84823bcee8834bd7b3f8c135a1db0693)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.256916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.270057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1270/2532] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:52.274082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.286224 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e4db9-5294-4d59-80a0-18efd344937c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.294022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.294022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.294022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.294022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.308712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.310429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.326665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84823bcee8834bd7b3f8c135a1db0693', 'request_id': 'req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:52.338136 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1271/2533] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:52.338345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84823bcee8834bd7b3f8c135a1db0693', 'request_id': 'req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:52.341603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.347326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84823bcee8834bd7b3f8c135a1db0693', 'request_id': 'req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:52.356158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334c56-33a2-4d20-9005-7d3afb914c77 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.365573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] The token cache is being invalidate because project 84823bcee8834bd7b3f8c135a1db0693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:29:52.365896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:52.366746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-aa33f1cb-c279-4b0b-a5d8-71ce0979891b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:29:52.368815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1263/2534] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:29:52 2022] DELETE /identity/v3/projects/84823bcee8834bd7b3f8c135a1db0693 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:52.373513 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420b138-03d7-4ae4-9788-3b85bcd40317 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.373826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420b138-03d7-4ae4-9788-3b85bcd40317 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.374107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420b138-03d7-4ae4-9788-3b85bcd40317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.397345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1420b138-03d7-4ae4-9788-3b85bcd40317 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:52.398632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1420b138-03d7-4ae4-9788-3b85bcd40317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:52.429884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.431010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.431010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.431200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] PATH_INFO: `/v3/projects/792bbab1f0034c3b9aa88621303c0f63/users/9b06e92e04954cbb852a1a856603ecae/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.448976 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792bbab1f0034c3b9aa88621303c0f63, user_id=9b06e92e04954cbb852a1a856603ecae, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.451933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029409e-e70c-4a47-8fb7-22bda30d5075 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.468943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1272/2535] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:52 2022] PUT /identity/v3/projects/792bbab1f0034c3b9aa88621303c0f63/users/9b06e92e04954cbb852a1a856603ecae/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:52.480811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.481550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.482241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.495057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1264/2536] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:52.503153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.503478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.503714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.517494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] MFA Rules not processed for user `9b06e92e04954cbb852a1a856603ecae`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:52.518991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:52.527000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:52.528489 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:52.557923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:52.598222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1265/2537] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:52.603059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1273/2538] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:29:52.603230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.610539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.612008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:52.612644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0bd18-d138-418b-8981-859cd17a5812 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.618690 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.619583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.619583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.619834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.620508 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:29:52.620822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.622599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.641108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1266/2539] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:29:52.642559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.654881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20559145-a9d0-4baa-a818-1f0b20922a96 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.661857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.662713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.663117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.663426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.664083 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.666500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.673717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:52.692112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080aab6-3142-4946-ba7c-e5bc3c0407f3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.700772 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.701735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.702236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.702491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.712818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.715234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.721701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1267/2540] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/projects => generated 355 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:52.724218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.726660 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1274/2541] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:29:52.823019 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac98a0e-403a-4115-8a87-0243920f04e6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.829108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.829929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.830158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.830441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.831483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.834735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.872786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1268/2542] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:52 2022] POST /identity/v3/users => generated 278 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:29:52.874154 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.888900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702056-ab00-48ad-a368-c75ebabff7f8 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.894414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.895076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.895354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.895623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.896402 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.898573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.917342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1275/2543] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:52.921601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.935507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4cf36-299c-4b8c-88a4-e4502060eabc admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.945071 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.946104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.946402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.946676 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:52.947816 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:52.949914 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f29f47-8777-4ed3-a4b5-ed2b5e557f39 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:52.961282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1269/2544] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:52.965354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:52.979381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea829748-cd7a-4010-a052-41bd4a4e4c47 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:52.987937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:52.989115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:52.989618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:52.990073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] PATH_INFO: `/v3/projects/ff8e3520d53149ca8cb429a5b984a12c/users/8b0d842dc92f4cbe8b5a92f44ece5a62/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.011383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8e3520d53149ca8cb429a5b984a12c, user_id=8b0d842dc92f4cbe8b5a92f44ece5a62, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.014646 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.041848 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1276/2545] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:52 2022] PUT /identity/v3/projects/ff8e3520d53149ca8cb429a5b984a12c/users/8b0d842dc92f4cbe8b5a92f44ece5a62/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:29:53.052471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.053144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.053557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.100540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] MFA Rules not processed for user `8b0d842dc92f4cbe8b5a92f44ece5a62`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:53.102869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:53.204442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1270/2546] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:53.209791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.285540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ec448-2377-457b-ab1c-cd6289e75d34 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.290719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.291453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.291799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.292254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.292909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.294836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.325015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1277/2547] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:29:53 2022] POST /identity/v3/projects => generated 357 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:53.329043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.340099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa6dfd-8d9f-4651-99fc-c69822b57563 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.395623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:53.404503 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.405131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.405730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.406009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.406654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.408770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.435562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1271/2548] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:29:53 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:29:53.439579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.453293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458d21f-4269-4def-94d7-33fbb1e11fe9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.459558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.460182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.460810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.460810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.461359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.462998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.474518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1278/2549] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:29:53.478938 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.489484 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8f408-da33-43ba-ba76-d16cf1990871 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.496216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.497099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.497389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.497716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] PATH_INFO: `/v3/projects/204478eb5e06462cbdf0abde699483e1/users/7beb1d46881644ed93509fabcd5db06f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.515552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204478eb5e06462cbdf0abde699483e1, user_id=7beb1d46881644ed93509fabcd5db06f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.518147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.533016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1272/2550] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:53 2022] PUT /identity/v3/projects/204478eb5e06462cbdf0abde699483e1/users/7beb1d46881644ed93509fabcd5db06f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:53.537989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.548424 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048bf20-dd9f-4829-b9bf-deaf2224c45f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.562079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:29:53.611945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.612576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.612883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.613150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.613852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.615952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956f570-b08b-4e00-b6b4-474cee259fbc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.627452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1279/2551] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:29:53.632426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.642672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ef655-d451-4198-aa6a-6741511f9027 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.647599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.648260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.648569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.648788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] PATH_INFO: `/v3/projects/204478eb5e06462cbdf0abde699483e1/users/7beb1d46881644ed93509fabcd5db06f/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.650832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204478eb5e06462cbdf0abde699483e1, user_id=7beb1d46881644ed93509fabcd5db06f, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.654293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.669981 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1273/2552] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:29:53 2022] PUT /identity/v3/projects/204478eb5e06462cbdf0abde699483e1/users/7beb1d46881644ed93509fabcd5db06f/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:29:53.700546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.700808 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.701079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.732993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] MFA Rules not processed for user `7beb1d46881644ed93509fabcd5db06f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:29:53.734580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:29:53.810658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1280/2553] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:29:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:29:53.824188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:53.895226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57188b-85b0-457b-826c-c62653176271 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:53.900004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:53.900623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:53.900925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:53.901200 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:53.912350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:53.913948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:53.924623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1274/2554] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:29:54.016889 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:29:54.027295 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2e8f3-f7ea-415b-b214-5856a0fee4f5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:29:54.031193 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:29:54.031828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:29:54.032113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:29:54.032386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:29:54.108178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:29:54.109769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:29:54.119088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1281/2555] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:29:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:00.659702 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:00.662143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:00.761089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0fd95-02b1-45b5-8203-0d7dc414b2a5 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:00.768151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:00.768977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:00.769357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:00.769727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:00.772762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:00.785888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:00.788865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7aee-19d7-4079-b4a4-6f39b226a1bf service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:00.800021 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1275/2556] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:01.367630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:01.369132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:01.412669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:01.439309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7077-3870-4d9d-a162-59675a935f6f service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:01.444520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:01.445148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:01.445429 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:01.445731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] PATH_INFO: `/v3/users/9b06e92e04954cbb852a1a856603ecae` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:01.446869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b06e92e04954cbb852a1a856603ecae)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:01.448785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:01.497521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b06e92e04954cbb852a1a856603ecae'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:01.505893 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b06e92e04954cbb852a1a856603ecae'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:01.518430 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b06e92e04954cbb852a1a856603ecae'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:01.519091 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b06e92e04954cbb852a1a856603ecae'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:01.527112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b06e92e04954cbb852a1a856603ecae'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:01.534004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1282/2557] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:01 2022] DELETE /identity/v3/users/9b06e92e04954cbb852a1a856603ecae => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:01.538694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:01.538985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:01.539256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:01.560028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:01.561253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:01.587317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:01.621809 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1276/2558] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:01.632192 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:01.633677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:01.690115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dabc06-ac69-4977-9d77-48ddacda74c6 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:01.694528 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:01.695124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:01.695423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:01.695692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:01.704580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:01.706032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976aa77-4628-4b02-8a7a-05749705abd4 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:01.715526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1283/2559] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:01.999094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.000757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:02.013955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8ce48-9ab6-4046-b162-26873e67067a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.018997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.019726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.020082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.020431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] PATH_INFO: `/v3/projects/792bbab1f0034c3b9aa88621303c0f63` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.022045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=792bbab1f0034c3b9aa88621303c0f63)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.024079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.080238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792bbab1f0034c3b9aa88621303c0f63', 'request_id': 'req-ac7bde64-1cab-4c4a-b51d-217e7925a67a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:02.089855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792bbab1f0034c3b9aa88621303c0f63', 'request_id': 'req-ac7bde64-1cab-4c4a-b51d-217e7925a67a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:02.098187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792bbab1f0034c3b9aa88621303c0f63', 'request_id': 'req-ac7bde64-1cab-4c4a-b51d-217e7925a67a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:02.111030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] The token cache is being invalidate because project 792bbab1f0034c3b9aa88621303c0f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:02.111316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:02.111985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ac7bde64-1cab-4c4a-b51d-217e7925a67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:02.113461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1277/2560] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:01 2022] DELETE /identity/v3/projects/792bbab1f0034c3b9aa88621303c0f63 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:02.119338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.119614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.119879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.142710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:02.143922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:02.207237 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:02.221148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1284/2561] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:02.229586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.229904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.230173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.253672 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:02.255531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:02.262682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1278/2562] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:02.266931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.278031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9dc0e-034c-4b89-be7b-242eb1b0a123 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.282922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.283623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.283960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.284304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.285025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:02.285404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.287811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.299888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1285/2563] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:02.305237 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.315075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947bea8-bcaa-4ae8-91f5-772278f37656 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.319901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.320558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.320857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.321125 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.321804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.323694 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.355791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1279/2564] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:02 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:02.360538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.371316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2444-b94f-4b5a-afca-3f9e589eee2e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.436718 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.437442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.437877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.438263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.438979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.440970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.472865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1286/2565] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:02 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:02.477003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.488890 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5d22f-88e4-467b-906b-8fe4e8f760c4 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.494785 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.495500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.495826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.496104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.496872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.498541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.511350 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1280/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:02.515420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.531172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba5a3-0259-41dd-aecf-de3f867c1771 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.540270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.541504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.542177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.542689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.544129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.546422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7e60-080c-435c-8805-a17dcfda631c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.563189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1287/2567] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:02.567510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.579598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f9bcb-8efd-4532-a647-7c54ddb1aa50 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.585887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.585887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.585887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.586176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] PATH_INFO: `/v3/projects/390efc66a4a54d33ad682e3022524588/users/f7925d08c85a404bbe6cba11ac339da4/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.602007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] RBAC: Authorizing `identity:create_grant(project_id=390efc66a4a54d33ad682e3022524588, user_id=f7925d08c85a404bbe6cba11ac339da4, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.604768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.619234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1281/2568] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:02 2022] PUT /identity/v3/projects/390efc66a4a54d33ad682e3022524588/users/f7925d08c85a404bbe6cba11ac339da4/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:02.631114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.631420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.631675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.659906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] MFA Rules not processed for user `f7925d08c85a404bbe6cba11ac339da4`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:02.661105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:02.725121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1288/2569] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:02.736091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:02.737845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:02.790969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:02.814465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca404-dd33-493b-8494-cb2fc07d438c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:02.820341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:02.820883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:02.821155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:02.821425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:02.829107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:02.830716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f935bce-5588-4205-875a-7975f54bfd7e service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:02.844306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1282/2570] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:03.070801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:03.125258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:03.139412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d564-c6cd-444b-9eed-b1844a345dd3 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:03.144027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:03.144665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:03.144944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:03.145216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] PATH_INFO: `/v3/users/f7925d08c85a404bbe6cba11ac339da4` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:03.146934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7925d08c85a404bbe6cba11ac339da4)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:03.148873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:03.198269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7925d08c85a404bbe6cba11ac339da4'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.206082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7925d08c85a404bbe6cba11ac339da4'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.219419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7925d08c85a404bbe6cba11ac339da4'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.220142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7925d08c85a404bbe6cba11ac339da4'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.228380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7925d08c85a404bbe6cba11ac339da4'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.234286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1289/2571] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:03 2022] DELETE /identity/v3/users/f7925d08c85a404bbe6cba11ac339da4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:03.238724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:03.239204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:03.239299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:03.258950 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:03.259903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:03.325154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1283/2572] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:03.337892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:03.339394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:03.402163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5db400-9f24-4d8b-b37f-b605498d8246 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:03.406215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:03.406951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:03.407230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:03.407501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:03.418361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:03.419875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba750fa0-8371-4582-95dd-af2378dae35d service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:03.433680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1290/2573] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:03.771949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:03.773732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:03.799357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410de5d-906c-44ed-8056-0a36eff92604 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:03.805791 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:03.806419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:03.806759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:03.807088 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] PATH_INFO: `/v3/projects/390efc66a4a54d33ad682e3022524588` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:03.808196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=390efc66a4a54d33ad682e3022524588)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:03.810288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:03.863861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390efc66a4a54d33ad682e3022524588', 'request_id': 'req-6d66fb73-5dea-4948-bce1-e412e210c057'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.873338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390efc66a4a54d33ad682e3022524588', 'request_id': 'req-6d66fb73-5dea-4948-bce1-e412e210c057'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.881602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390efc66a4a54d33ad682e3022524588', 'request_id': 'req-6d66fb73-5dea-4948-bce1-e412e210c057'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.893486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] The token cache is being invalidate because project 390efc66a4a54d33ad682e3022524588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:03.893802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.894634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6d66fb73-5dea-4948-bce1-e412e210c057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:03.897144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1284/2574] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:03 2022] DELETE /identity/v3/projects/390efc66a4a54d33ad682e3022524588 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:03.902300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:03.902833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:03.903103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:03.925181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:03.926447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:03.998309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1291/2575] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:04.007222 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.007532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.007799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.029836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:04.031226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:04.039860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1285/2576] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:04.044094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.054733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3922c-b51d-488a-aa71-5c89d38a8708 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.059436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.060045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.060322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.060591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.061238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:04.061537 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.063203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.072973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:04.077129 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1292/2577] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:04.082687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.094106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e82cd-b431-42df-95d7-67f6cd3b46d0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.099519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.100270 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.100383 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.100613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.101259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.103255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.139500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1286/2578] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:04.143907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.154840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207c9d-957c-4d23-b2ad-af02f0a7fbcf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.213083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.213719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.214002 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.214276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.214926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.216857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.237803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1293/2579] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:04.242095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.253186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf698f-3973-4120-8e2e-a94069b6e245 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.258969 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.259625 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.259904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.260173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.260822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.262494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.274045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1287/2580] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:04.278377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.288641 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7af74-0895-4ac9-8d91-af6f5903e814 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.293288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.293993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.294275 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.294573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.295251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.296836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3bd48-0952-475c-a265-0a1984409ebd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.307639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1294/2581] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:04.311925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.322661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb1fe9-1343-44d1-add2-541ddc2962b7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.327358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.328018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.328302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.328582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] PATH_INFO: `/v3/projects/ac41e4ec6ed84341952c5d14ce649a99/users/8c6a61337a0d49b19db1fdddbf442c3f/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.345376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac41e4ec6ed84341952c5d14ce649a99, user_id=8c6a61337a0d49b19db1fdddbf442c3f, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.348038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.351517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:04.363663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1288/2582] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:04 2022] PUT /identity/v3/projects/ac41e4ec6ed84341952c5d14ce649a99/users/8c6a61337a0d49b19db1fdddbf442c3f/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:04.373519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.373838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.374156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.403463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] MFA Rules not processed for user `8c6a61337a0d49b19db1fdddbf442c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:04.404735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:04.468622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1295/2583] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:04.473099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.543122 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d24d5a-b375-460f-86ad-db22db007e51 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.548550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.549231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.549725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.550123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.550812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.553251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.582097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1289/2584] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:04.587321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.602926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e76bf-21f2-4f38-95b0-17dd399baa5d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.660439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.661049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.661344 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.662050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.662268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.664172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.681304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:04.692142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1296/2585] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:04.697352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.708135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92152bd8-961a-48b8-8a27-be55a87da252 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.713155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.713794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.714074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.714370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.715005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.716593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.727555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1290/2586] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:04.731841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.744220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c7bb9-538d-446c-a216-7bcf5f80e509 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.750520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.751217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.751498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.751778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] PATH_INFO: `/v3/projects/614316434437485293696075870d1ade/users/790dd468170f485c9986993581a59643/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.769152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614316434437485293696075870d1ade, user_id=790dd468170f485c9986993581a59643, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.771994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.788381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1297/2587] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:04 2022] PUT /identity/v3/projects/614316434437485293696075870d1ade/users/790dd468170f485c9986993581a59643/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:04.792314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.793875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.794476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:04.863371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1e8d-05f0-4a4b-9fe3-76cf5520d580 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.865985 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7a56a-5844-4a11-a25e-0b33b9922cc2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.874818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.875911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.876227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.876755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.877705 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.879895 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.894571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1298/2588] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:04.895587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:04.906845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c11b9-36b3-4d20-8232-bb1a0e301cb2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:04.912521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.912920 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.913200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.913472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] PATH_INFO: `/v3/projects/614316434437485293696075870d1ade/users/790dd468170f485c9986993581a59643/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.915432 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614316434437485293696075870d1ade, user_id=790dd468170f485c9986993581a59643, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:04.918204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea88b3-0c6c-41dc-b6df-d62b3c658703 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:04.921481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:04.921869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.922153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.922429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.924171 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:04.929737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1299/2589] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:04 2022] PUT /identity/v3/projects/614316434437485293696075870d1ade/users/790dd468170f485c9986993581a59643/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:04.941027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:04.941317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:04.941599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:04.970830 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:04.971049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] MFA Rules not processed for user `790dd468170f485c9986993581a59643`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:04.972238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:05.003796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:05.005280 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:05.018127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1291/2590] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:05.061037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1300/2591] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:05.080297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:05.081505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:05.157108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3aff-3116-4105-a1c7-35c4a48c4d07 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:05.211344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:05.211952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:05.212225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:05.212492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:05.229593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:05.231144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38844f84-b3e7-44d2-9f27-9732d0916a68 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:05.241751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1292/2592] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:05.325441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:05.336656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48540e1-c716-48cd-9bae-1e30e5897fb9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:05.340466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:05.341087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:05.341363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:05.341632 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:05.383719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:05.403227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:05.404686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:05.415358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1301/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:07.468304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:07.468789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:07.468882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:07.531009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:07.532229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:07.584130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:07.603900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1293/2594] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:07.655863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:07.657273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:07.749446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996015-2762-46a8-805c-10bd9a54eb65 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:07.754971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:07.755856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:07.756201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:07.756473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:07.764997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:07.766595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012873-e51b-4450-8fb5-4f47804b74d9 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:07.776416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1302/2595] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 12:30:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 12:30:08.174670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:08.247248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833845c-9b45-43ff-a608-066ebe0367b1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:08.252106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:08.252755 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.253036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.253309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] PATH_INFO: `/v3/users/8c6a61337a0d49b19db1fdddbf442c3f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.254524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6a61337a0d49b19db1fdddbf442c3f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:08.257196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:08.316076 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6a61337a0d49b19db1fdddbf442c3f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:08.316826 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6a61337a0d49b19db1fdddbf442c3f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:08.330907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6a61337a0d49b19db1fdddbf442c3f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:08.340802 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6a61337a0d49b19db1fdddbf442c3f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:08.341500 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-9bd75ce0-1643-4fa0-9eb5-c2141a942fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6a61337a0d49b19db1fdddbf442c3f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:08.346234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.346530 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.346806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.354762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1294/2596] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:08 2022] DELETE /identity/v3/users/8c6a61337a0d49b19db1fdddbf442c3f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:08.359114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.359389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.359653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.369414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:08.370719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.383282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:08.384515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.435101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1303/2597] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 6 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:08.449057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:08.450764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.453487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1295/2598] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:08.464141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:08.466204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.478023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:08.521323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:08.540199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc1357-7112-4369-9fff-0fac1e0b1864 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:08.545068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7a8b3-de20-457a-9d91-ca2afd49b7e5 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:08.547826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:08.548653 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.548829 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:08.549578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.549812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.549975 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.550098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.551047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.562219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:08.563735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:08.564534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab373d-7b3e-4b2a-b2ee-b845295d76a0 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:08.565510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:08.574301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1304/2599] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 12:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 12:30:08.577822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1296/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:08.838919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.839505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.839505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.861083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:08.862388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.870589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1305/2601] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 6 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:08.887149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:08.904009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670426e-86f7-48f8-bd66-48ea2b42cc1e service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:08.910132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:08.910798 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.911045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.911932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.922212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:08.925405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:08.925902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:08.927494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:08.937822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1297/2602] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 12:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 12:30:08.950495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f354c-871e-4312-bc2c-ca76d507908e None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:08.965169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:08.965868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:08.966320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:08.966962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] PATH_INFO: `/v3/projects/ac41e4ec6ed84341952c5d14ce649a99` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:08.968109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac41e4ec6ed84341952c5d14ce649a99)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:08.970271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.040711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac41e4ec6ed84341952c5d14ce649a99', 'request_id': 'req-2afa2853-c354-4334-abc3-8fa83f0da955'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.049244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac41e4ec6ed84341952c5d14ce649a99', 'request_id': 'req-2afa2853-c354-4334-abc3-8fa83f0da955'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.062991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac41e4ec6ed84341952c5d14ce649a99', 'request_id': 'req-2afa2853-c354-4334-abc3-8fa83f0da955'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.078297 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] The token cache is being invalidate because project ac41e4ec6ed84341952c5d14ce649a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:09.078593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.079267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-2afa2853-c354-4334-abc3-8fa83f0da955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.080928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1306/2603] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:08 2022] DELETE /identity/v3/projects/ac41e4ec6ed84341952c5d14ce649a99 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:09.085435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:09.098259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31bc3-c908-4d51-9e16-fbed5a010ad1 service swift] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:09.182865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:09.183527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.184061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.184474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] PATH_INFO: `/v3/users/790dd468170f485c9986993581a59643` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.185736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=790dd468170f485c9986993581a59643)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:09.188120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.243574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790dd468170f485c9986993581a59643'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.244297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790dd468170f485c9986993581a59643'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.262295 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790dd468170f485c9986993581a59643'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.283151 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790dd468170f485c9986993581a59643'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.283828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790dd468170f485c9986993581a59643'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.298570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1298/2604] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:09 2022] DELETE /identity/v3/users/790dd468170f485c9986993581a59643 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:09.353222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.353514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.353829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.391106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] MFA Rules not processed for user `2ab3501f173341538094ffeb2a59f004`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:09.392214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.392322 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:09.393759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.435632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:09.490274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1307/2605] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 6 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:09.497150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816b73-aaba-4a09-b93d-3fdf24a10c98 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:09.502564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:09.503188 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:09.503331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.503458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.503726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] PATH_INFO: `/v3/users/8b0d842dc92f4cbe8b5a92f44ece5a62` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.505393 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0d842dc92f4cbe8b5a92f44ece5a62)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:09.505856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.507629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.577910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0d842dc92f4cbe8b5a92f44ece5a62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.582261 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0d842dc92f4cbe8b5a92f44ece5a62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.601282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0d842dc92f4cbe8b5a92f44ece5a62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.610314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0d842dc92f4cbe8b5a92f44ece5a62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.611281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3bde9689-3099-4b55-9cdb-d608378df75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0d842dc92f4cbe8b5a92f44ece5a62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.625674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122b0c-862e-48d5-b43e-e62803a1e56d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:09.629687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1299/2606] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:09 2022] DELETE /identity/v3/users/8b0d842dc92f4cbe8b5a92f44ece5a62 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:09.633976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.634327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.634644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.637303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7848ff43a794bd4aa35bdff59a53c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:09.638269 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.638558 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.638874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.642438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.654968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:09.657973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:09.659108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.659970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:09.662793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.674777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1308/2607] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 6 12:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 12:30:09.700743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:09.707183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.725866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b65f2-02cf-48d0-8018-8929bf6f2dc6 service swift] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:09.742032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1300/2608] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:09.751905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:09.753402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:09.805560 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:09.806264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.806540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.806836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] PATH_INFO: `/v3/projects/614316434437485293696075870d1ade` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.807925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:09.808175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] RBAC: Authorizing `identity:delete_project(project_id=614316434437485293696075870d1ade)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:09.811690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.832642 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3f46-2665-4c5a-ae89-335a52a3256b None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:09.840169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:09.842314 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.842621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.842896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.854366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:09.855831 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538a39-45dd-4896-be46-9dbbfdeb01ec service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:09.871944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1301/2609] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:09.880924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614316434437485293696075870d1ade', 'request_id': 'req-b1892668-404b-4daf-a006-d73869fa690c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.887688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614316434437485293696075870d1ade', 'request_id': 'req-b1892668-404b-4daf-a006-d73869fa690c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.897797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614316434437485293696075870d1ade', 'request_id': 'req-b1892668-404b-4daf-a006-d73869fa690c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.910931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] The token cache is being invalidate because project 614316434437485293696075870d1ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:09.912538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.913214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b1892668-404b-4daf-a006-d73869fa690c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:09.914764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1309/2610] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:09 2022] DELETE /identity/v3/projects/614316434437485293696075870d1ade => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:09.923397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:09.923692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:09.923972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:09.953315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:09.955043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.071999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1302/2611] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:10.080240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.080591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.080935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.113854 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:10.116241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.132161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1310/2612] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:10.136195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.148011 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613a0a4-4784-461f-8119-64dcafde41e1 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.153255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.154054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.154332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.154733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.155389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:10.155692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.157309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.175276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1303/2613] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:10.180502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.205601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e933c-952b-4a50-80fc-cb8ebd1157b7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.217684 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.218333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.218629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.218902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.219594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.224926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.235175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.236972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.252108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef1a4-8234-4204-b6cf-3f6365bd0112 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.258669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.259531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.259884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.260297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] PATH_INFO: `/v3/projects/ff8e3520d53149ca8cb429a5b984a12c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.261535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8e3520d53149ca8cb429a5b984a12c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.263705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.266939 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1311/2614] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:10.272389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.329427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8e3520d53149ca8cb429a5b984a12c', 'request_id': 'req-7cd5881b-e123-4b37-b651-2600db29d1be'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.341009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8e3520d53149ca8cb429a5b984a12c', 'request_id': 'req-7cd5881b-e123-4b37-b651-2600db29d1be'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.347778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8e3520d53149ca8cb429a5b984a12c', 'request_id': 'req-7cd5881b-e123-4b37-b651-2600db29d1be'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.352057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:10.356239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a168635-8318-4a0b-a714-31385258b75f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.361402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] The token cache is being invalidate because project ff8e3520d53149ca8cb429a5b984a12c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:10.361761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.362600 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.365451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1304/2615] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:10 2022] DELETE /identity/v3/projects/ff8e3520d53149ca8cb429a5b984a12c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:10.369185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.370812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.428971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.429743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.430142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.430439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.431685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.433350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.442578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:10.466525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5881b-e123-4b37-b651-2600db29d1be admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.469146 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1312/2616] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:10.471501 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.472060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.472708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.472990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.473262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] PATH_INFO: `/v3/users/7beb1d46881644ed93509fabcd5db06f` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.473362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.475053 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7beb1d46881644ed93509fabcd5db06f)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.477095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.486940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ad4a9-1dbe-442a-b362-8c9c62d0d778 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.492758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.493503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.493945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.494247 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.495000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.499796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.512654 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1313/2617] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:10.517760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.534026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b20f-c1f5-4b96-921d-ed5152b4918f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.542943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7beb1d46881644ed93509fabcd5db06f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.543562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.543764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.543887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7beb1d46881644ed93509fabcd5db06f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.544051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.544318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.544948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.546580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.558604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1314/2618] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:10.560640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7beb1d46881644ed93509fabcd5db06f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.562547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.564929 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.570722 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7beb1d46881644ed93509fabcd5db06f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.571453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-09f72a80-163b-484b-a79e-4f752c6d97b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7beb1d46881644ed93509fabcd5db06f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:10.580531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d2f80-c93a-46f6-8aa6-662f502510d7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.587368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.587604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1305/2619] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:10 2022] DELETE /identity/v3/users/7beb1d46881644ed93509fabcd5db06f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:10.588036 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.588317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.588619 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] PATH_INFO: `/v3/projects/69b837fbde32418bb719956aa972972a/users/a09ef00db9e44e6c8a71db4d5f7d2a86/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.609195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b837fbde32418bb719956aa972972a, user_id=a09ef00db9e44e6c8a71db4d5f7d2a86, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.609195 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.637470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1315/2620] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:10 2022] PUT /identity/v3/projects/69b837fbde32418bb719956aa972972a/users/a09ef00db9e44e6c8a71db4d5f7d2a86/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:10.639886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.640219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.640502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.678510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] MFA Rules not processed for user `a09ef00db9e44e6c8a71db4d5f7d2a86`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:10.680133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.768264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1306/2621] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:10.776082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.864968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ac79-f735-4803-9964-c61a3dddc440 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:10.870815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:10.871475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:10.871765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:10.872111 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:10.872845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:10.875277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:10.941274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1316/2622] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:10.943712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.944774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:10.946173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:10.972866 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da35bc5-4911-4b1b-9087-fd2c070c783e None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.023342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.053528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.054213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.054847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.055123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] PATH_INFO: `/v3/projects/204478eb5e06462cbdf0abde699483e1` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.056745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=204478eb5e06462cbdf0abde699483e1)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.059138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.070160 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.089779 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862bfa-80a9-4266-b3e0-da42cc150320 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.097364 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.098202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.098486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.099541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.100318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.102431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.127585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204478eb5e06462cbdf0abde699483e1', 'request_id': 'req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:11.130988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1317/2623] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:10 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:11.135044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.148966 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15645096-b259-4441-b9ae-f91a7f32200e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.152141 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204478eb5e06462cbdf0abde699483e1', 'request_id': 'req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:11.159531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204478eb5e06462cbdf0abde699483e1', 'request_id': 'req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:11.175098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] The token cache is being invalidate because project 204478eb5e06462cbdf0abde699483e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:11.175397 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:11.176339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fc8eed9b-ceb2-44c2-9618-3f993aed78cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:11.179126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1307/2624] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:10 2022] DELETE /identity/v3/projects/204478eb5e06462cbdf0abde699483e1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:11.184703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7283b0-851c-4a4b-a4f2-a3458db49dfa None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.184838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7283b0-851c-4a4b-a4f2-a3458db49dfa None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.185105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7283b0-851c-4a4b-a4f2-a3458db49dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.209708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bc7283b0-851c-4a4b-a4f2-a3458db49dfa None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:11.211100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bc7283b0-851c-4a4b-a4f2-a3458db49dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.225498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.226397 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.226721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.227075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.227862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.229625 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.240787 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1318/2625] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:11.246037 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.250559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.271308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52941ec-1b6a-4b02-b850-2860ba5e8d00 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.276675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.277810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.277967 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.278254 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] PATH_INFO: `/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96/users/65cd32066542430c9ebdfb0aec2e2b37/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.297167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1308/2626] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:11.297896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4c2acf4ddc4d02b2b83ebdabbe0e96, user_id=65cd32066542430c9ebdfb0aec2e2b37, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.300404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.305627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.306050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.306321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.316015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1319/2627] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:11 2022] PUT /identity/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96/users/65cd32066542430c9ebdfb0aec2e2b37/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:11.320392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.331861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:11.333796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.389382 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19684097-8134-4b47-961f-d8d63d6c7172 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.394289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.394897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.395177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.396162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.397359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.398157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1309/2628] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:11.399165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.402845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.413443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1320/2629] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:11.413788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03f0d0-fed0-44c6-bd2d-8e92ff247ffe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.414487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.418471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.419127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.419378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.419645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.420288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:11.420584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.422209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.425532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6612c-5589-4ce7-b10e-a427016e92a1 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.430994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.431648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.432404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.432686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] PATH_INFO: `/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96/users/65cd32066542430c9ebdfb0aec2e2b37/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.434688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4c2acf4ddc4d02b2b83ebdabbe0e96, user_id=65cd32066542430c9ebdfb0aec2e2b37, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.434987 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1310/2630] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:11.437370 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227a2ec-620b-4a91-9f87-56f1ebb49dbd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.439216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.448761 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1321/2631] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:11 2022] PUT /identity/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96/users/65cd32066542430c9ebdfb0aec2e2b37/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:11.449041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1a067-5747-4c9f-b4e9-63305732906a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.460751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.461045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.461324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.462292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.491426 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] MFA Rules not processed for user `65cd32066542430c9ebdfb0aec2e2b37`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:11.492649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.498008 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.510568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.511194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.511457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.511725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.512359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.514304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.543271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1311/2632] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:11.547310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.569466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1322/2633] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:11.580340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.588875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.612675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae628a-baa5-4ad5-80a7-245611570cc9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.617774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.618387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.618673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.618949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.619611 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.621555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.644386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1312/2634] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:11.649735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.654393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2ad7e-3b80-425e-bc4b-7868e1888e7f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.660498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e347d-5f06-4846-bdf6-da1ace51ca44 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.665857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.666457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.666744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.667022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.667670 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.669287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.679639 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1313/2635] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:11.683386 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.693800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe13a-1671-4d0c-95b3-d70f106c7ce0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.698833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.699424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.699769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.700045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.700686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.702316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.705544 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.706300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.706582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.706856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.714018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1314/2636] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:11.718381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.728121 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef747f-9f0b-4bf6-8f9e-ed1c481f3cc3 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.733111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.733786 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.734063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.734334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] PATH_INFO: `/v3/projects/44f93f4ce4784aea84aa59059bf6b7af/users/f68b8313a0354d359dee97df7bf7d67c/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.749947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f93f4ce4784aea84aa59059bf6b7af, user_id=f68b8313a0354d359dee97df7bf7d67c, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.752645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6e17f-2996-46c7-89f7-8a26010c3a4f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.766974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1315/2637] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:11 2022] PUT /identity/v3/projects/44f93f4ce4784aea84aa59059bf6b7af/users/f68b8313a0354d359dee97df7bf7d67c/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:11.772521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.774057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.779154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.779450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.779720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.784677 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1323/2638] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:11.808297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] MFA Rules not processed for user `f68b8313a0354d359dee97df7bf7d67c`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:11.809509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.848871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.870994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.881892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590d478-75d1-43e1-81d1-7d64f4624730 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.886001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1316/2639] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:11.887329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:11.890770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:11.934731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.935337 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.935609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.935874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.937674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:11.957372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4963-fd68-43b8-b9ef-6b3685a2486d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:11.962780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:11.963409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:11.963707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:11.963977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:11.964616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:11.966595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:11.995250 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1317/2640] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:11.999583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.003047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.004497 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.015419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1324/2641] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:12.076783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2af988-966f-4db1-849b-20c685166b3d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.083480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.085536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.085854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.086132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.086783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.088751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.102521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.104490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.123000 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1318/2642] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:11 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:12.124046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.125046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501e0e0-bea5-4a00-950c-30bff780b17d service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.132476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.133133 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.133542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.133945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] PATH_INFO: `/v3/users/71dcf98c074847158fabdb71a7594902` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.134585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5cfc-9d6e-4c58-848b-a3854891a2c2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.135504 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71dcf98c074847158fabdb71a7594902)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.137535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.139652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.140338 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.140680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.141162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.142082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.144483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.167719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1319/2643] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:12.171645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.186046 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353fcda-cb42-4dd6-82b1-0078d9d1ee89 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.202155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.202155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.202155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.202155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] PATH_INFO: `/v3/projects/846d154cdca943299a048bcd0326fa2f/users/b9d72a59687c4c76963b6c1d8874d454/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.208944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71dcf98c074847158fabdb71a7594902'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.215825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] RBAC: Authorizing `identity:create_grant(project_id=846d154cdca943299a048bcd0326fa2f, user_id=b9d72a59687c4c76963b6c1d8874d454, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.218001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71dcf98c074847158fabdb71a7594902'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.218935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.234377 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71dcf98c074847158fabdb71a7594902'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.237682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1320/2644] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:12 2022] PUT /identity/v3/projects/846d154cdca943299a048bcd0326fa2f/users/b9d72a59687c4c76963b6c1d8874d454/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:12.237858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71dcf98c074847158fabdb71a7594902'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.240111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.244981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9eb0047d-305f-47ca-896a-be6cf8aed3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71dcf98c074847158fabdb71a7594902'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.251351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1325/2645] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:12 2022] DELETE /identity/v3/users/71dcf98c074847158fabdb71a7594902 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:12.257936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.258338 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.258716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.264932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:12.281683 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:12.282960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.321509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf1ef-9bd2-4bd2-b17a-54adcc8c296a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.337351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:12.393907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1326/2646] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:12.404417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.405935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.417772 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.418552 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.418970 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.419248 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.425621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.428588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.445695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1321/2647] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:12.448176 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.450197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.464324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb5758-57fa-46ca-8970-1e5611a3343a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.469309 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.469995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.470272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.470541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] PATH_INFO: `/v3/projects/846d154cdca943299a048bcd0326fa2f/users/b9d72a59687c4c76963b6c1d8874d454/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.472464 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=846d154cdca943299a048bcd0326fa2f, user_id=b9d72a59687c4c76963b6c1d8874d454, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.475617 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f11ec-6d77-4070-b7fa-0a027092e73a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.477533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7a7bf-b228-4430-af72-5df0e2b8af71 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.481386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.482139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.482416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.482687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.485922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1322/2648] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:12 2022] PUT /identity/v3/projects/846d154cdca943299a048bcd0326fa2f/users/b9d72a59687c4c76963b6c1d8874d454/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:12.492686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.494175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.496431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.496716 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.496990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.504554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1327/2649] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:12.527822 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] MFA Rules not processed for user `b9d72a59687c4c76963b6c1d8874d454`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:12.529045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.600839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1323/2650] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:12.612876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.614346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.676478 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4746-2df0-404e-a848-41b84a0d1be3 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.683892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.685408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.731601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:12.738206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.738826 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.739112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.740159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.751044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.752521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.762399 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1328/2651] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:12.765449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dedc5-e071-4e97-a72f-3c23c628b3f6 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.770097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.771230 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.771506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.771874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] PATH_INFO: `/v3/projects/26fe7ddc88334bc6a7912c5fa07b9516` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.773068 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26fe7ddc88334bc6a7912c5fa07b9516)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.775070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.810599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:12.812071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.823383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde94ff-35fb-4a59-909b-b78acd2f5cb9 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:12.824629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fe7ddc88334bc6a7912c5fa07b9516', 'request_id': 'req-651fe33f-13f2-4a08-9b03-b07f8f912487'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.832418 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fe7ddc88334bc6a7912c5fa07b9516', 'request_id': 'req-651fe33f-13f2-4a08-9b03-b07f8f912487'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.838840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fe7ddc88334bc6a7912c5fa07b9516', 'request_id': 'req-651fe33f-13f2-4a08-9b03-b07f8f912487'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.850252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] The token cache is being invalidate because project 26fe7ddc88334bc6a7912c5fa07b9516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:12.850960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.851626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:12.853157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1324/2652] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:12 2022] DELETE /identity/v3/projects/26fe7ddc88334bc6a7912c5fa07b9516 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:12.867777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:12.886571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:12.887243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:12.887550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:12.887746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:12.889587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:12.960203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:12.961600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570755d-1145-4eec-89a3-7fe845602980 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:12.972009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1329/2653] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:16.380836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:16.382293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:16.454685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe33f-13f2-4a08-9b03-b07f8f912487 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:16.459616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:16.460241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:16.460519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:16.460793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] PATH_INFO: `/v3/users/f68b8313a0354d359dee97df7bf7d67c` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:16.461984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68b8313a0354d359dee97df7bf7d67c)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:16.463918 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:16.511473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68b8313a0354d359dee97df7bf7d67c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:16.512132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68b8313a0354d359dee97df7bf7d67c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:16.524091 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:16.525478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68b8313a0354d359dee97df7bf7d67c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:16.542524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68b8313a0354d359dee97df7bf7d67c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:16.543353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68b8313a0354d359dee97df7bf7d67c'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:16.557358 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1325/2654] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:16 2022] DELETE /identity/v3/users/f68b8313a0354d359dee97df7bf7d67c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:16.562160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:16.562721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:16.562721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:16.582526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:16.584272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:16.647527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1330/2655] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:16.657491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:16.659415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:16.720728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9d100-837e-4ecd-b95d-8bd86238b00b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:16.724995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:16.725589 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:16.725894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:16.726163 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:16.738841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:16.740304 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96162321-6500-4482-9b5d-7c5aeadaa82d service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:16.751968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1326/2656] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:16.921157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:16.922722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:16.935101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234690-e4d4-4715-8b80-99ec766c773c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:16.939937 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:16.940579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:16.941098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:16.941376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] PATH_INFO: `/v3/users/a09ef00db9e44e6c8a71db4d5f7d2a86` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:16.943320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09ef00db9e44e6c8a71db4d5f7d2a86)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:16.945281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:16.995291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09ef00db9e44e6c8a71db4d5f7d2a86'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.003563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09ef00db9e44e6c8a71db4d5f7d2a86'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.021563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09ef00db9e44e6c8a71db4d5f7d2a86'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.023368 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09ef00db9e44e6c8a71db4d5f7d2a86'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.032406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09ef00db9e44e6c8a71db4d5f7d2a86'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.038521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1331/2657] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:16 2022] DELETE /identity/v3/users/a09ef00db9e44e6c8a71db4d5f7d2a86 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:17.043228 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.043629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.043994 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.045793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.047534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.059986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:17.065867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:17.067743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.119713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9d59-e61d-46ae-94b1-861f86ac0044 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.124831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:17.125457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.125760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.126032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] PATH_INFO: `/v3/projects/44f93f4ce4784aea84aa59059bf6b7af` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.127208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f93f4ce4784aea84aa59059bf6b7af)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:17.129092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:17.132553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1327/2658] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:17.142458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.143954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.178829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f93f4ce4784aea84aa59059bf6b7af', 'request_id': 'req-3e204ed6-0900-4677-96ad-16aa4e5a114a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.184413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f93f4ce4784aea84aa59059bf6b7af', 'request_id': 'req-3e204ed6-0900-4677-96ad-16aa4e5a114a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.191900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f93f4ce4784aea84aa59059bf6b7af', 'request_id': 'req-3e204ed6-0900-4677-96ad-16aa4e5a114a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.204885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7d95-fb8a-4f19-b1ae-867b08a21d6a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.205827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] The token cache is being invalidate because project 44f93f4ce4784aea84aa59059bf6b7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:17.206148 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.206796 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.208618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1332/2659] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:17 2022] DELETE /identity/v3/projects/44f93f4ce4784aea84aa59059bf6b7af => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:17.212351 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.213860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.246519 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:17.265183 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:17.266294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.266675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.267205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.269029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.285836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204ed6-0900-4677-96ad-16aa4e5a114a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.291242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:17.291874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.292173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.292448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] PATH_INFO: `/v3/users/b9d72a59687c4c76963b6c1d8874d454` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.293758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d72a59687c4c76963b6c1d8874d454)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:17.295693 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:17.338385 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:17.338627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:17.340121 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:17.350842 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1328/2660] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:17.351173 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d72a59687c4c76963b6c1d8874d454'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.358606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d72a59687c4c76963b6c1d8874d454'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.371487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d72a59687c4c76963b6c1d8874d454'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.372214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d72a59687c4c76963b6c1d8874d454'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.379331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d72a59687c4c76963b6c1d8874d454'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.385959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1333/2661] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:17 2022] DELETE /identity/v3/users/b9d72a59687c4c76963b6c1d8874d454 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:17.673952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.675973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.699559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533b35-f52e-4f9f-90f9-ab999f8a4f1f service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.709031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.710699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.778234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:17.778896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.779166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.779439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] PATH_INFO: `/v3/projects/69b837fbde32418bb719956aa972972a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.780583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b837fbde32418bb719956aa972972a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:17.782577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:17.782940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199592d-3ec3-4b3d-9a27-24b7db3631dc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.787823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:17.788454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.788734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.789004 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] PATH_INFO: `/v3/projects/846d154cdca943299a048bcd0326fa2f` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.790157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=846d154cdca943299a048bcd0326fa2f)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:17.792064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:17.831323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b837fbde32418bb719956aa972972a', 'request_id': 'req-98e4af4a-16ae-423e-8d70-acd9213b8fd8'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.839258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b837fbde32418bb719956aa972972a', 'request_id': 'req-98e4af4a-16ae-423e-8d70-acd9213b8fd8'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.844270 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846d154cdca943299a048bcd0326fa2f', 'request_id': 'req-050fca54-90d3-4435-9ca8-6e5664a15e9d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.845327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b837fbde32418bb719956aa972972a', 'request_id': 'req-98e4af4a-16ae-423e-8d70-acd9213b8fd8'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.850359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846d154cdca943299a048bcd0326fa2f', 'request_id': 'req-050fca54-90d3-4435-9ca8-6e5664a15e9d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.857319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] The token cache is being invalidate because project 69b837fbde32418bb719956aa972972a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:17.857601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.858384 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846d154cdca943299a048bcd0326fa2f', 'request_id': 'req-050fca54-90d3-4435-9ca8-6e5664a15e9d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.858543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.860066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1329/2662] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:17 2022] DELETE /identity/v3/projects/69b837fbde32418bb719956aa972972a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:17.863907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:17.865566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.872290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] The token cache is being invalidate because project 846d154cdca943299a048bcd0326fa2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:17.872640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.873569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-050fca54-90d3-4435-9ca8-6e5664a15e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:17.875479 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1334/2663] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:17 2022] DELETE /identity/v3/projects/846d154cdca943299a048bcd0326fa2f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:17.881553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42627-c45c-4f26-9a2b-033be63c0dcf None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.881930 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42627-c45c-4f26-9a2b-033be63c0dcf None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.882276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42627-c45c-4f26-9a2b-033be63c0dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:17.897252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:17.904043 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-15a42627-c45c-4f26-9a2b-033be63c0dcf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:17.905552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-15a42627-c45c-4f26-9a2b-033be63c0dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:17.946084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4af4a-16ae-423e-8d70-acd9213b8fd8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:17.949797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:17.971124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1335/2664] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:17.980398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:17.980679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:17.980964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.002383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:18.003622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.011757 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.012443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.012581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1336/2665] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:18.013804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.014130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] PATH_INFO: `/v3/users/65cd32066542430c9ebdfb0aec2e2b37` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.015445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=65cd32066542430c9ebdfb0aec2e2b37)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.016820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.017610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.027245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c6435-4c01-4056-8155-76f1c9777f0f None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.032249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.033105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.033105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.033374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.034059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:18.034363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.036624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.049585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1337/2666] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 12:30:18.053897 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.064318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e3ef-e394-4087-91e7-d55a41fc3dc6 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.069853 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.070603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.071025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.071202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65cd32066542430c9ebdfb0aec2e2b37'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.071323 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.072006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.072129 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65cd32066542430c9ebdfb0aec2e2b37'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.073945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.086733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65cd32066542430c9ebdfb0aec2e2b37'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.095909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65cd32066542430c9ebdfb0aec2e2b37'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.096426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65cd32066542430c9ebdfb0aec2e2b37'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.104555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1338/2667] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:18.108566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.110223 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.111886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1330/2668] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:17 2022] DELETE /identity/v3/users/65cd32066542430c9ebdfb0aec2e2b37 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:18.188262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe87e-2a75-4101-86c1-85a424d2cfb7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.247973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.248597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.248874 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.249147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.249823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.251757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.280845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1339/2669] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:18.285022 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.296646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eb8d3-9daf-416a-8389-b6ff9aca5c6c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.301939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.302635 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.303008 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.303542 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.304118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.305982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.319954 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1331/2670] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:18.321716 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.332395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1e9a5-8839-4f72-8a51-3f35e8ddbef9 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.338171 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.338881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.339233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.339600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.340328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.342060 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.353565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1340/2671] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:18.357619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.368451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0a6-0448-4005-8f39-c80e5f739b30 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.373260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.373956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.374238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.374514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] PATH_INFO: `/v3/projects/6c1030f0042a4196bb7ef420a8e1bbd5/users/ac0dd5b91f344b5c852111a4b002dbd3/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.390177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1030f0042a4196bb7ef420a8e1bbd5, user_id=ac0dd5b91f344b5c852111a4b002dbd3, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.392850 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a6ea-4de7-4675-8ba4-266297659556 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.410486 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1332/2672] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:18 2022] PUT /identity/v3/projects/6c1030f0042a4196bb7ef420a8e1bbd5/users/ac0dd5b91f344b5c852111a4b002dbd3/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:18.412618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.414512 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.420762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.421058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.421330 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.431056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0c597-f436-4251-bbee-9420bbd669b2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.445797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:18.456992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] MFA Rules not processed for user `ac0dd5b91f344b5c852111a4b002dbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:18.459762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.461296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:18.494253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.494799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.495076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.495365 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] PATH_INFO: `/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.496507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d4c2acf4ddc4d02b2b83ebdabbe0e96)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.498598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.532887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1333/2673] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:18.539403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.553673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ae970-d241-4854-a82f-4a164b005d35 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.553857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d4c2acf4ddc4d02b2b83ebdabbe0e96', 'request_id': 'req-e638a673-2d35-4fad-9279-5113105c628e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.559444 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d4c2acf4ddc4d02b2b83ebdabbe0e96', 'request_id': 'req-e638a673-2d35-4fad-9279-5113105c628e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.567798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d4c2acf4ddc4d02b2b83ebdabbe0e96', 'request_id': 'req-e638a673-2d35-4fad-9279-5113105c628e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.579771 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] The token cache is being invalidate because project 5d4c2acf4ddc4d02b2b83ebdabbe0e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:18.580068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.580721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e638a673-2d35-4fad-9279-5113105c628e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:18.582468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1341/2674] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:18 2022] DELETE /identity/v3/projects/5d4c2acf4ddc4d02b2b83ebdabbe0e96 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:18.587759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43467e-2984-4de8-876a-a04e9d7b4f3f None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.588046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43467e-2984-4de8-876a-a04e9d7b4f3f None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.588325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43467e-2984-4de8-876a-a04e9d7b4f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.608046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6e43467e-2984-4de8-876a-a04e9d7b4f3f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:18.609267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6e43467e-2984-4de8-876a-a04e9d7b4f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.612569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.613296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.613575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.613916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.614771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.616841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.617823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1342/2675] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:18.625860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.626550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.626824 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.649238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1334/2676] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:18.650373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:18.651829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.653664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.655227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:18.716673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1343/2677] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:18.721018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7543ab7-8d33-41d8-9445-d13f180db694 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.721453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.726637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.727251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.727529 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.727800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.728441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.730427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.731169 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052f0b7-ed34-49bc-9c02-243e7b0910b8 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.735885 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.736490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.736773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.737048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.737729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:18.738038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.740473 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.753737 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1344/2678] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 12:30:18.757506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1335/2679] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:18.758478 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.758605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.768644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c3833-ee49-4b43-9da5-57ced5a8936a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.768833 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6b31a-422d-463c-b7a9-9f0a60d67426 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.773955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.774109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.774561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.774674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.774841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.774957 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.775112 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.775226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.775747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.775909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.777663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.777946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.790241 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1345/2680] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:18.794050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.797967 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:18.804425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2cdcb-954e-41cb-9aa9-299060a96d44 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.809538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.810248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.810580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.810846 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] PATH_INFO: `/v3/projects/7656bd4978ab4b58b0540807e9bb04d9/users/f6a1f8ca351a4691ade34cc7905a77bb/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.811614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1336/2681] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:18.815923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.826983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7656bd4978ab4b58b0540807e9bb04d9, user_id=f6a1f8ca351a4691ade34cc7905a77bb, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.829727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.844681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1346/2682] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:18 2022] PUT /identity/v3/projects/7656bd4978ab4b58b0540807e9bb04d9/users/f6a1f8ca351a4691ade34cc7905a77bb/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:18.849498 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.855383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:18.883777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e066-c3d6-4126-a11f-11c489aa749c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.919207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d45f30-b50b-419b-a84c-7fb229547fe6 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.923971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.924582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.924859 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.925126 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.925801 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.927457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.937843 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1347/2683] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:18.942047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.946331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.946973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.947252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.947525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.948158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.950114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.951852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49564a3b-bd95-4430-a42c-91b689a72ef2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:18.956583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:18.957236 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.957513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.958422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] PATH_INFO: `/v3/projects/7656bd4978ab4b58b0540807e9bb04d9/users/f6a1f8ca351a4691ade34cc7905a77bb/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.960484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7656bd4978ab4b58b0540807e9bb04d9, user_id=f6a1f8ca351a4691ade34cc7905a77bb, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:18.963194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623add5-1b6c-47be-b383-a191f023b39c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:18.972887 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1337/2684] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:18.975799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1348/2685] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:18 2022] PUT /identity/v3/projects/7656bd4978ab4b58b0540807e9bb04d9/users/f6a1f8ca351a4691ade34cc7905a77bb/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:18.984647 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:18.986356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:18.986640 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:18.986931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:18.988124 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a6a86-92d0-462c-897c-65246d73a2e8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.053780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.054305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.054606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.054877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.055546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.057214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.068895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1338/2686] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:19.073233 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.088042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714168f-74b4-4657-b40b-c4de0502e6e7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.093671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.094458 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.094638 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.094927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.095568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.097199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.109769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1339/2687] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:19.113719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.123943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e5b6-cd27-496d-99f4-b98a68e3943a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.128593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.129232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.129505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.129982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] PATH_INFO: `/v3/projects/35a6c5d626cf4baaa164bbab0508714d/users/cc98a7f6a0d44240a8c053f5197681aa/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.138387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] MFA Rules not processed for user `f6a1f8ca351a4691ade34cc7905a77bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:19.139608 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.142032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:19.145817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a6c5d626cf4baaa164bbab0508714d, user_id=cc98a7f6a0d44240a8c053f5197681aa, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.148615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc3af-a706-4e51-b9ef-0820595e1bba admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.163003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1340/2688] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:19 2022] PUT /identity/v3/projects/35a6c5d626cf4baaa164bbab0508714d/users/cc98a7f6a0d44240a8c053f5197681aa/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:19.174196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.174471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.174741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.204477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] MFA Rules not processed for user `cc98a7f6a0d44240a8c053f5197681aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:19.205735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.214207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1349/2689] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:19.225408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.226844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.273923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1341/2690] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:19.278666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.297509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6ca3-8552-4745-99d9-c8c3f2d20c38 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.324325 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:19.343293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb61d2-e6f1-4ca0-93c5-cb2d4c0d4936 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.348257 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.348936 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.349427 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.349614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.349732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.349928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.350041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.351059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.351165 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.352733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.382431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1342/2691] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:19 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:19.386497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.414157 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.415595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.425546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1350/2692] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:19.450390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d25d-b25b-4e19-b89b-b341ed6db3eb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.455168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.455777 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.456051 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.456334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.456945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.459149 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.470365 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:19.473118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.475172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.483762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1343/2693] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:19 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:19.486539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3530-05b5-426d-8382-5cf1cd67ad05 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.487554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.498570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f0f0-9083-4743-9404-e162733ecd0d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.503275 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.503877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.504155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.504426 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.505731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.507379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.517349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1344/2694] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:19.522306 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.532417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e906d9a-f257-4e59-880a-919162c16ea6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.538315 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.538499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.539063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.539173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.539363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.539470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.539615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.539724 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] PATH_INFO: `/v3/projects/550ddea125bf46d893c63a8f3074c648/users/2dc24d4946854903b0ae17786c268370/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.541381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.555540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550ddea125bf46d893c63a8f3074c648, user_id=2dc24d4946854903b0ae17786c268370, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.559507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.575794 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1345/2695] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:19 2022] PUT /identity/v3/projects/550ddea125bf46d893c63a8f3074c648/users/2dc24d4946854903b0ae17786c268370/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:19.579416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.605721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.607165 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.618901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1351/2696] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:19.652618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8b55-cfa0-4586-8018-876903a1af10 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.658800 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.659708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.660194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.660349 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.662181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.664024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f67f-c810-4b6c-8dbc-0781e2bc9c62 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.676805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1346/2697] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:19.681883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.693499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddd4ef-20cc-4b9f-ad4e-81ca3966524e service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.698438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.699110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.699388 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.699656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] PATH_INFO: `/v3/projects/550ddea125bf46d893c63a8f3074c648/users/2dc24d4946854903b0ae17786c268370/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.701590 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] RBAC: Authorizing `identity:create_grant(project_id=550ddea125bf46d893c63a8f3074c648, user_id=2dc24d4946854903b0ae17786c268370, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.704273 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.716105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1352/2698] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:19 2022] PUT /identity/v3/projects/550ddea125bf46d893c63a8f3074c648/users/2dc24d4946854903b0ae17786c268370/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:19.727195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.727744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.727844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.758079 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] MFA Rules not processed for user `2dc24d4946854903b0ae17786c268370`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:19.759447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:19.841498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1347/2699] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:19.853794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:19.879207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:19.928872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dcff0-0937-46a6-86cd-0e52c1ed333c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:19.932681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:19.933314 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:19.933594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:19.933902 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:19.944791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:19.946306 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:19.956531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1353/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:20.044346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:20.055038 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3cae4-5225-4a96-bd80-4ae9c7805258 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:20.058650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:20.059246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:20.059521 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:20.059789 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:20.102685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:20.120161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:20.121608 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ef0e1-3c76-4a4b-bfb8-b23498338aa0 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:20.131601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1348/2701] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:20.414299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:20.478217 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fab0a-3a98-454c-9374-7ad63a995e2d service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:20.482754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:20.483379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:20.483656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:20.483922 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] PATH_INFO: `/v3/users/cc98a7f6a0d44240a8c053f5197681aa` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:20.485034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc98a7f6a0d44240a8c053f5197681aa)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:20.486989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:20.535353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc98a7f6a0d44240a8c053f5197681aa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:20.542335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc98a7f6a0d44240a8c053f5197681aa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:20.555360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc98a7f6a0d44240a8c053f5197681aa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:20.556017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc98a7f6a0d44240a8c053f5197681aa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:20.562776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc98a7f6a0d44240a8c053f5197681aa'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:20.569055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1354/2702] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:20 2022] DELETE /identity/v3/users/cc98a7f6a0d44240a8c053f5197681aa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:20.573556 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:20.573939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:20.574223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:20.593979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:20.595196 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:20.658009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1349/2703] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:20.669423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:20.670980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:20.718785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:20.732047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19cf56-a2c7-4a2b-a060-f1fcc173d08b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:20.735725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:20.736333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:20.736607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:20.736876 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:20.747242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:20.748673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:20.759250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1355/2704] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:21.032205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:21.034232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:21.055515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce63c0-1266-40ab-ac44-99ce082f4ef3 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:21.060824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:21.061461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.061779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.062041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] PATH_INFO: `/v3/projects/35a6c5d626cf4baaa164bbab0508714d` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.063208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a6c5d626cf4baaa164bbab0508714d)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:21.065187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:21.121043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a6c5d626cf4baaa164bbab0508714d', 'request_id': 'req-fd3417fa-a347-4427-99c6-9f0d119929da'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.133606 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a6c5d626cf4baaa164bbab0508714d', 'request_id': 'req-fd3417fa-a347-4427-99c6-9f0d119929da'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.140372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a6c5d626cf4baaa164bbab0508714d', 'request_id': 'req-fd3417fa-a347-4427-99c6-9f0d119929da'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.154182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] The token cache is being invalidate because project 35a6c5d626cf4baaa164bbab0508714d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:21.154585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.155242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.156947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1350/2705] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:21 2022] DELETE /identity/v3/projects/35a6c5d626cf4baaa164bbab0508714d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:21.161229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:21.171049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38499-f5a3-409c-b0b8-c097c58deef2 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:21.232228 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:21.232867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.233145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.233417 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] PATH_INFO: `/v3/users/2dc24d4946854903b0ae17786c268370` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.234597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc24d4946854903b0ae17786c268370)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:21.236522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:21.284627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc24d4946854903b0ae17786c268370'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.295159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc24d4946854903b0ae17786c268370'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.307783 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc24d4946854903b0ae17786c268370'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.308551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc24d4946854903b0ae17786c268370'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.315712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0c1baa42-ff0d-4981-90d7-591f6c61b916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc24d4946854903b0ae17786c268370'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.322581 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1356/2706] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:21 2022] DELETE /identity/v3/users/2dc24d4946854903b0ae17786c268370 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:21.622278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:21.624155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:21.638512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3417fa-a347-4427-99c6-9f0d119929da admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:21.712960 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:21.714039 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.714339 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.714624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] PATH_INFO: `/v3/projects/550ddea125bf46d893c63a8f3074c648` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.715782 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550ddea125bf46d893c63a8f3074c648)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:21.718860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:21.728771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:21.775366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550ddea125bf46d893c63a8f3074c648', 'request_id': 'req-067ae588-8a54-4289-8960-fde7639fc545'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.784933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550ddea125bf46d893c63a8f3074c648', 'request_id': 'req-067ae588-8a54-4289-8960-fde7639fc545'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.792006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550ddea125bf46d893c63a8f3074c648', 'request_id': 'req-067ae588-8a54-4289-8960-fde7639fc545'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.804392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] The token cache is being invalidate because project 550ddea125bf46d893c63a8f3074c648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:21.804683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.805341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-067ae588-8a54-4289-8960-fde7639fc545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:21.806847 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1351/2707] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:21 2022] DELETE /identity/v3/projects/550ddea125bf46d893c63a8f3074c648 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:21.811865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.812145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.812418 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.833580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:21.840174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:21.902658 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1357/2708] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:21.910624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.911623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.912082 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.932208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:21.933407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:21.941283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1352/2709] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:21.945690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:21.951924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:21.956076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fec96-5737-4c54-95d2-57488aa3b001 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:21.962775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:21.963550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.963834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.964278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:21.964971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:21.965425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:21.967073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:21.979386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1358/2710] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:21.984266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:21.993744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f40870-ff3e-4544-b530-2c91aa584518 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:21.998593 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:21.999198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:21.999476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:21.999748 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.000378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.002335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.032457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1353/2711] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:21 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:22.037488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.047680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389de6d-2bb5-4c05-82b9-d86ecdd8be29 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.108776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.109559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.110006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.110484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.111183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.114017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.135886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1359/2712] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:22 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:22.141162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.152928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab82450-0882-489c-8a11-e8e3d05aa7f0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.157551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.158189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.158462 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.158727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.159370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.161605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.171817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1354/2713] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:22.176055 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.186573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ad075-1ed8-4060-a552-0c19abb0b1d7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.191106 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.191698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.192462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.192723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.193363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.194990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d5699-d436-4494-8fff-31b8137d2538 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.205681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1360/2714] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:22.210626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.220756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af17529-3687-4aa6-95ba-06ebbfde062c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.226067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.226713 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.226990 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.227267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] PATH_INFO: `/v3/projects/1f31426fbbc14e35a6bdc17b4a8a39a8/users/36379fff22c9457a8945deb179a3820e/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.242453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f31426fbbc14e35a6bdc17b4a8a39a8, user_id=36379fff22c9457a8945deb179a3820e, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.245110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.259274 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1355/2715] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:22 2022] PUT /identity/v3/projects/1f31426fbbc14e35a6bdc17b4a8a39a8/users/36379fff22c9457a8945deb179a3820e/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:22.271094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.271396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.271742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.299208 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] MFA Rules not processed for user `36379fff22c9457a8945deb179a3820e`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:22.300410 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:22.364719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1361/2716] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:22.369355 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.440128 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deab25-3226-4198-a819-fe322f7fef3b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.445490 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.446144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.446425 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.446733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.447378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.449287 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.476666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1356/2717] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:22 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:22.480741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.491744 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e3b1a-0b60-4791-83d0-803121606840 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.548142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.548746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.549024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.549321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.549992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.552439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.560604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:22.575376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1362/2718] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:22 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:22.579405 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.589905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b92be-2d1d-48cd-99c4-8f4caf7ceb66 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.594578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.595177 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.595463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.595736 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.596380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.598025 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.609013 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1357/2719] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:22.613159 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.624119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42381d-632b-4f58-986d-5b2b078ebdb4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.628998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.630412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.630836 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.630960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] PATH_INFO: `/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf/users/53ff091ab26646cd9f3da99336ecd5d3/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.647584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cd7481e1b0644b38a31f0a620fc0ebf, user_id=53ff091ab26646cd9f3da99336ecd5d3, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.650304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.663277 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1363/2720] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:22 2022] PUT /identity/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf/users/53ff091ab26646cd9f3da99336ecd5d3/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:22.667271 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.673803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:22.678677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57754e-e8c5-43c4-a5f6-d0832a3cce11 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.738551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.739742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.740317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.740317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.740946 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.742588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022f3c1-63c2-4f38-bd20-13a5b51f6d98 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.752705 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1358/2721] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:22.758303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.769222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32899dd-51f5-4067-849a-cf14dc8ec3aa admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.774872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.775535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.775810 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.776080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] PATH_INFO: `/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf/users/53ff091ab26646cd9f3da99336ecd5d3/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.778292 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cd7481e1b0644b38a31f0a620fc0ebf, user_id=53ff091ab26646cd9f3da99336ecd5d3, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:22.780907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:22.791990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1364/2722] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:22 2022] PUT /identity/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf/users/53ff091ab26646cd9f3da99336ecd5d3/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:22.802897 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.803186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.803465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:22.831516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] MFA Rules not processed for user `53ff091ab26646cd9f3da99336ecd5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:22.832957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:22.906773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1359/2723] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:22.918071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:22.919811 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:22.980250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be168a-227c-4ab9-851c-1953babd86bf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:22.983988 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:22.984612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:22.984892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:22.985143 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.032592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:23.034147 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:23.047794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:23.049328 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f561-4441-4ada-9ca6-cdb1076a9c09 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:23.058532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1365/2724] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:23.103590 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b7f7b-b947-44df-8abe-bd38436e09ed None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:23.108288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:23.108909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:23.109190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:23.109453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] PATH_INFO: `/v3/users/ac0dd5b91f344b5c852111a4b002dbd3` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.110619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0dd5b91f344b5c852111a4b002dbd3)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:23.112583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:23.142567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:23.162584 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0dd5b91f344b5c852111a4b002dbd3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.163449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0dd5b91f344b5c852111a4b002dbd3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.176977 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0dd5b91f344b5c852111a4b002dbd3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.184793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0dd5b91f344b5c852111a4b002dbd3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.185825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0dd5b91f344b5c852111a4b002dbd3'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.200368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1360/2725] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:23 2022] DELETE /identity/v3/users/ac0dd5b91f344b5c852111a4b002dbd3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:23.205367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:23.205899 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:23.206069 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.227984 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:23.229191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:23.302574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1366/2726] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:23.310727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:23.312448 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:23.381626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf834c-4fed-4039-9b30-76b7a9494a4f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:23.385424 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:23.386058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:23.386331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:23.386597 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.396406 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:23.397868 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:23.409026 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1361/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:23.686672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:23.688285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:23.704909 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26dfe-4707-4e5e-96c3-df57473aeeed None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:23.710868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:23.711623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:23.711898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:23.712145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] PATH_INFO: `/v3/projects/6c1030f0042a4196bb7ef420a8e1bbd5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.713264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1030f0042a4196bb7ef420a8e1bbd5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:23.715276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:23.716912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:23.792050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1030f0042a4196bb7ef420a8e1bbd5', 'request_id': 'req-05f3eb5b-749c-4165-bc53-e21136a96e93'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.800875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1030f0042a4196bb7ef420a8e1bbd5', 'request_id': 'req-05f3eb5b-749c-4165-bc53-e21136a96e93'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.812219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1030f0042a4196bb7ef420a8e1bbd5', 'request_id': 'req-05f3eb5b-749c-4165-bc53-e21136a96e93'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.831322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] The token cache is being invalidate because project 6c1030f0042a4196bb7ef420a8e1bbd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:23.831713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.834481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.837122 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1367/2728] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:23 2022] DELETE /identity/v3/projects/6c1030f0042a4196bb7ef420a8e1bbd5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:23.841455 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:23.851972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a184-644c-4446-84e5-269986cd2b76 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:23.908941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:23.909561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:23.909863 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:23.910135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] PATH_INFO: `/v3/users/f6a1f8ca351a4691ade34cc7905a77bb` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:23.911871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a1f8ca351a4691ade34cc7905a77bb)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:23.913856 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:23.960774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a1f8ca351a4691ade34cc7905a77bb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.961432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a1f8ca351a4691ade34cc7905a77bb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.963444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:23.976773 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a1f8ca351a4691ade34cc7905a77bb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.983902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a1f8ca351a4691ade34cc7905a77bb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.984493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-a096ad11-d225-421a-b1ac-08df1b726797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a1f8ca351a4691ade34cc7905a77bb'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:23.996561 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1362/2729] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:23 2022] DELETE /identity/v3/users/f6a1f8ca351a4691ade34cc7905a77bb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:24.281606 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.283113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:24.298166 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3eb5b-749c-4165-bc53-e21136a96e93 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.362063 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.362959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.363249 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.363524 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] PATH_INFO: `/v3/projects/7656bd4978ab4b58b0540807e9bb04d9` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.365552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7656bd4978ab4b58b0540807e9bb04d9)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.367643 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.419350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7656bd4978ab4b58b0540807e9bb04d9', 'request_id': 'req-895da5be-d4c3-47d2-9803-43e9ab50009d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:24.431752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7656bd4978ab4b58b0540807e9bb04d9', 'request_id': 'req-895da5be-d4c3-47d2-9803-43e9ab50009d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:24.440991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7656bd4978ab4b58b0540807e9bb04d9', 'request_id': 'req-895da5be-d4c3-47d2-9803-43e9ab50009d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:24.453870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] The token cache is being invalidate because project 7656bd4978ab4b58b0540807e9bb04d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:24.454290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:24.454942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-895da5be-d4c3-47d2-9803-43e9ab50009d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:24.456422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1368/2730] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:24 2022] DELETE /identity/v3/projects/7656bd4978ab4b58b0540807e9bb04d9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:24.461362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.461900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.462018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.482336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:24.483754 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:24.550204 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1363/2731] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:24.559212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.559647 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.559993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.580907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:24.582333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:24.593428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1369/2732] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:24.597859 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.608036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd771b-99aa-44cf-92ee-3c56e2d3a798 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.612680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.613277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.613554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.613896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.614537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:24.614836 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.616437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.628258 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1364/2733] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:24.633748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.643142 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ab67-cd02-476a-9375-8f5706284313 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.648678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.649285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.649563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.649863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.650502 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.652644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.664206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:24.681962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1370/2734] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:24 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:24.686113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.696783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8d86-d994-4ff2-bb71-e02e172437c6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.755408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.755865 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.756144 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.756416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.757058 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.759132 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.780943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1365/2735] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:24 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:24.785392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.796845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226cd6-36d4-474d-9b79-92508362f1e4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.802913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.803638 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.803934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.804256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.804964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.806678 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.817971 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1371/2736] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:24.824084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.835211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b611532-2f7e-49a8-836e-e3ff246cb28d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.840684 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.841299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.841577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.841883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.842532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.844595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd915a3-da30-4800-ad80-31f5bb33116a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.856469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1366/2737] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:24.860817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:24.873044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1099f-497c-4946-925b-bbfa2516cca4 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:24.881304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:24.882147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.882534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.882857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] PATH_INFO: `/v3/projects/605080f0f80e4c51a85b4a52ea2b2716/users/0cdb8dfb58d34923a386d1b934307684/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.899221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=605080f0f80e4c51a85b4a52ea2b2716, user_id=0cdb8dfb58d34923a386d1b934307684, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:24.902109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:24.932566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1372/2738] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:24 2022] PUT /identity/v3/projects/605080f0f80e4c51a85b4a52ea2b2716/users/0cdb8dfb58d34923a386d1b934307684/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:24.943669 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62888448-1117-44fc-bc86-e4f738095fce None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:24.943952 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62888448-1117-44fc-bc86-e4f738095fce None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:24.944229 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62888448-1117-44fc-bc86-e4f738095fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:24.972879 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-62888448-1117-44fc-bc86-e4f738095fce None None] MFA Rules not processed for user `0cdb8dfb58d34923a386d1b934307684`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:24.974173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-62888448-1117-44fc-bc86-e4f738095fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:24.995454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:25.044256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1367/2739] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:25.056022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.057794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.117605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba692cca-bb0f-448d-95de-e2438146b09d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.121258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.121838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.122113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.122381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.131851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.133289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dddf303-34a5-4098-830b-90ee0bf748b3 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.142431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1373/2740] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:25.163253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62888448-1117-44fc-bc86-e4f738095fce None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.228428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62888448-1117-44fc-bc86-e4f738095fce None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.233184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.233840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.234118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.234384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] PATH_INFO: `/v3/users/0cdb8dfb58d34923a386d1b934307684` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.235517 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cdb8dfb58d34923a386d1b934307684)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.237481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.288846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cdb8dfb58d34923a386d1b934307684'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.288846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cdb8dfb58d34923a386d1b934307684'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.301210 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cdb8dfb58d34923a386d1b934307684'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.308931 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cdb8dfb58d34923a386d1b934307684'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.309557 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cdb8dfb58d34923a386d1b934307684'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.322451 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1368/2741] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:25 2022] DELETE /identity/v3/users/0cdb8dfb58d34923a386d1b934307684 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:25.328354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.328900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.329359 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.362006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:25.364731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.431888 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1374/2742] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:25.441925 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.444067 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.497988 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:25.512882 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82f60a-e651-4556-8456-5391245142ea admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.516693 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.517303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.517581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.517888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.528080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.529518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5b25-88a0-4361-b1c2-8387467c2fce service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.540902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1369/2743] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:25.633340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.634806 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.647542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73314679-2b1d-419c-b1aa-cef6db65ae82 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.652231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.653121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.653396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.653689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] PATH_INFO: `/v3/users/36379fff22c9457a8945deb179a3820e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.654822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36379fff22c9457a8945deb179a3820e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.656745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.719748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36379fff22c9457a8945deb179a3820e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.733862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36379fff22c9457a8945deb179a3820e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.735415 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:25.750584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36379fff22c9457a8945deb179a3820e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.751312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36379fff22c9457a8945deb179a3820e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.758729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36379fff22c9457a8945deb179a3820e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:25.765178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1375/2744] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:25 2022] DELETE /identity/v3/users/36379fff22c9457a8945deb179a3820e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:25.770674 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.771029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.771377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.791202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:25.792579 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.831256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.833515 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.874682 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1370/2745] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:25.885664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:25.887104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:25.917162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b892-a1b5-40be-bd84-a2a5c82f46d5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.922792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.923579 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.923926 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.924289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] PATH_INFO: `/v3/projects/605080f0f80e4c51a85b4a52ea2b2716` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.925627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=605080f0f80e4c51a85b4a52ea2b2716)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.927593 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.951628 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a42bf-10d5-48a8-b5d7-7fc6238ec749 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:25.955934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:25.956666 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:25.956815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:25.957073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:25.968858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:25.970353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8345f-4fe9-4ed4-9a87-cb9f65592800 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:25.985398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1371/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:26.000495 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605080f0f80e4c51a85b4a52ea2b2716', 'request_id': 'req-42c7d4c6-7bf8-4765-bf17-d566946df730'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.007457 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605080f0f80e4c51a85b4a52ea2b2716', 'request_id': 'req-42c7d4c6-7bf8-4765-bf17-d566946df730'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.015733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605080f0f80e4c51a85b4a52ea2b2716', 'request_id': 'req-42c7d4c6-7bf8-4765-bf17-d566946df730'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.029392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] The token cache is being invalidate because project 605080f0f80e4c51a85b4a52ea2b2716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:26.029733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.030437 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-42c7d4c6-7bf8-4765-bf17-d566946df730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.031952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1376/2747] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:25 2022] DELETE /identity/v3/projects/605080f0f80e4c51a85b4a52ea2b2716 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:26.037242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.037537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.037861 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.059249 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:26.060516 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.131581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1372/2748] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:26.139428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.139711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.139983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.162690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:26.163905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.171952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1377/2749] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:26.176718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.187033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1990e6f-b29c-4504-85c7-253850eb0c12 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.192563 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.195742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.196291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.196687 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.197368 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:26.198154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.199815 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.212586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1373/2750] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:26.217423 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.228007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc43b9-a15f-4b68-abfd-edc1279b843d None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.232766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.233374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.233670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.233949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.234577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.236472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.273090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1378/2751] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:26 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:26.277291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.281913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.283803 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.290089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc7f7-55b5-4783-b19b-f2f3271e72f8 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.306421 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:26.324774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:26.358780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcba012-7cca-41d5-be0f-a07f2fc6cc68 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.363521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.365011 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.365354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.365722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] PATH_INFO: `/v3/projects/1f31426fbbc14e35a6bdc17b4a8a39a8` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.366321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.366934 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.367079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f31426fbbc14e35a6bdc17b4a8a39a8)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.367227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.367487 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.368120 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.369230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.370185 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.393194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1374/2752] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:26 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:26.398752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.409357 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bbda4-3e51-4501-a04d-7ddc844fdc67 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.419115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f31426fbbc14e35a6bdc17b4a8a39a8', 'request_id': 'req-0e5c7882-9bd9-4500-8f07-4b1b0334323a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.425030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f31426fbbc14e35a6bdc17b4a8a39a8', 'request_id': 'req-0e5c7882-9bd9-4500-8f07-4b1b0334323a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.433794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f31426fbbc14e35a6bdc17b4a8a39a8', 'request_id': 'req-0e5c7882-9bd9-4500-8f07-4b1b0334323a'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.445745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] The token cache is being invalidate because project 1f31426fbbc14e35a6bdc17b4a8a39a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:26.446079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.446739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.448230 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1379/2753] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:26 2022] DELETE /identity/v3/projects/1f31426fbbc14e35a6bdc17b4a8a39a8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:26.452141 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.453427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.472276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.472876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.473219 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.473623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.474298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.475896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.488440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1375/2754] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:26.490599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.492326 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.505473 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972d5ff-2bcd-4773-9785-35fb247d515e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.510444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.511047 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.511323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.511594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.512242 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.513876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.523546 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7882-9bd9-4500-8f07-4b1b0334323a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.524833 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1376/2755] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:30:26.528973 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.529108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.529809 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.530226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.530508 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] PATH_INFO: `/v3/users/53ff091ab26646cd9f3da99336ecd5d3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.531661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ff091ab26646cd9f3da99336ecd5d3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.533591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.539337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c290f-2ac5-4f63-a38a-dbb1b4e73117 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.544017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.544663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.544944 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.545218 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] PATH_INFO: `/v3/projects/47ee5e0c9d7b45f8b2e10a338c20ea01/users/8e6a8778a7f44ace9927ca6ec474158d/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.561182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ee5e0c9d7b45f8b2e10a338c20ea01, user_id=8e6a8778a7f44ace9927ca6ec474158d, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.564016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e14994-1006-488b-b5c7-a633f48afc3b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.578127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1377/2756] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:26 2022] PUT /identity/v3/projects/47ee5e0c9d7b45f8b2e10a338c20ea01/users/8e6a8778a7f44ace9927ca6ec474158d/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:26.588083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ff091ab26646cd9f3da99336ecd5d3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.589729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.590123 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.590454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.592936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ff091ab26646cd9f3da99336ecd5d3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.602009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:26.608104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ff091ab26646cd9f3da99336ecd5d3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.609539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ff091ab26646cd9f3da99336ecd5d3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.617507 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ff091ab26646cd9f3da99336ecd5d3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:26.619876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] MFA Rules not processed for user `8e6a8778a7f44ace9927ca6ec474158d`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:26.621092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.623741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1380/2757] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:26 2022] DELETE /identity/v3/users/53ff091ab26646cd9f3da99336ecd5d3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:26.697345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:26.710733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1378/2758] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:30:26.722436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.724483 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:26.806280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3108bb-8551-44a9-a1fd-b952dc5471e5 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:26.809947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7739336ec72248feb11d1d6a7a8de166', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:26.810550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:26.810820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:26.811082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:26.819691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:26.821121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e45163-326a-418c-94ee-fe4a97da5db4 service glance] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:26.830856 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1381/2759] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:26.923390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:26.924824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:27.007470 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bb88e-916d-4ce0-a6bb-3f06890a2984 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.012530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.013255 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.014033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.014564 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] PATH_INFO: `/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.016161 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cd7481e1b0644b38a31f0a620fc0ebf)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.018857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.080276 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cd7481e1b0644b38a31f0a620fc0ebf', 'request_id': 'req-52b9a3ee-3b93-4283-ba28-d01ccddbc157'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:27.090817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cd7481e1b0644b38a31f0a620fc0ebf', 'request_id': 'req-52b9a3ee-3b93-4283-ba28-d01ccddbc157'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:27.097414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cd7481e1b0644b38a31f0a620fc0ebf', 'request_id': 'req-52b9a3ee-3b93-4283-ba28-d01ccddbc157'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:27.110347 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] The token cache is being invalidate because project 7cd7481e1b0644b38a31f0a620fc0ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:27.110645 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:27.111389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-52b9a3ee-3b93-4283-ba28-d01ccddbc157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:27.113007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1379/2760] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:26 2022] DELETE /identity/v3/projects/7cd7481e1b0644b38a31f0a620fc0ebf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:27.118471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.118748 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.119034 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.141567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:27.142959 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:27.213262 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1382/2761] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:27.224152 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.224465 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.224805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.245408 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:27.257225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:27.266686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1380/2762] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:27.270983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.283427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea62fc-6e76-4106-aa82-251336512933 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.288947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.289612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.289924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.290190 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.291149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:27.291539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.293290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.313576 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1383/2763] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:27.314909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.325286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e820f5-bda0-4051-a008-81d86cadad81 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.330155 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.330768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.331048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.331321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.331961 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.335166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.365949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1381/2764] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:27.370431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.381911 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cdac3-872a-4b65-90ea-414b7777572a admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.440071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.440702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.440979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.441245 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.442057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.444449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.465387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1384/2765] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:30:27.469422 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.480832 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ccc7-828b-4b7d-935b-a7afab5a3782 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.485476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.486104 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.486384 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.486677 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.487316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.490106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.503721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1382/2766] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:27.505420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.513760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:27.518386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d1775-0f76-4de8-9384-58e9c0aacc0b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.525316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.526163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.526598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.527147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.528081 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.529766 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2c017-691d-41c9-8520-d217a65aa114 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.544952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1385/2767] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:27.546348 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.560710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb64c0-ad91-4b57-8055-d8fb4c7f9f26 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.565417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.566135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.566483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.566779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] PATH_INFO: `/v3/projects/cd16aee0d02e443fa2573def6a246ab4/users/da5b83f300c544bc956e2e1ad1fcc8bd/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.583353 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd16aee0d02e443fa2573def6a246ab4, user_id=da5b83f300c544bc956e2e1ad1fcc8bd, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.586142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.599664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1383/2768] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:27 2022] PUT /identity/v3/projects/cd16aee0d02e443fa2573def6a246ab4/users/da5b83f300c544bc956e2e1ad1fcc8bd/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:27.610290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.610574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.610842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.640545 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] MFA Rules not processed for user `da5b83f300c544bc956e2e1ad1fcc8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:27.641837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:27.712822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1386/2769] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:27.717698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.745581 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:27.786804 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f21e69-f903-4219-b030-60e03ecc2c47 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.791968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.792604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.792889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.793153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.793828 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.795756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.823621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1384/2770] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:27.827724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.841913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f7a7a-045a-4c01-adbe-942483ac6198 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.905026 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.905639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.905949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.906220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.906896 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.908815 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.933201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1387/2771] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:27 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:27.937370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.949387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bee8d-9963-4ab2-9712-44ebb3fcf236 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.955010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.955631 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.955912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.956184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.956837 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.959277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:27.972632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1385/2772] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:27.977150 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:27.987477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ff2c9-7dc8-4e3e-82ca-484cd4171170 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:27.992838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:27.993523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:27.994022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:27.994335 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:27.995153 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:27.997098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1b9b6-7291-4e22-9f23-08e578f5ad08 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.012616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1388/2773] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:28.014142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.025179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd250bc5-8cef-41f0-8fac-5a1778e79233 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.030512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.031167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.031445 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.031714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] PATH_INFO: `/v3/projects/2712f86c0b34449cb3bc8fa0647325f5/users/709da4bf48a04f7fa2fcbd31ffa1d558/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.048515 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2712f86c0b34449cb3bc8fa0647325f5, user_id=709da4bf48a04f7fa2fcbd31ffa1d558, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.051291 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.065594 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1386/2774] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:28 2022] PUT /identity/v3/projects/2712f86c0b34449cb3bc8fa0647325f5/users/709da4bf48a04f7fa2fcbd31ffa1d558/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:28.077628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.078023 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.078263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.115598 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] MFA Rules not processed for user `709da4bf48a04f7fa2fcbd31ffa1d558`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:28.117019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:28.176813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:28.199455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1389/2775] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:28.204414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.286224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa33f01-3d64-408d-888d-626251f32474 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.290998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.291604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.291909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.292170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.292930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.294963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.324474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1387/2776] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:28 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:28.328513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.342533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636a901-ad9b-4959-9cc2-55a0e075eb30 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.411389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.412073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.412201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.412435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.413101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.415068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.438549 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1390/2777] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:28 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:28.445435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.458993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d95fd-9b55-473f-acaf-0aa4b9166542 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.463661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.464265 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.464538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.464803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.465439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.467086 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.478217 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1388/2778] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:28.482616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.493019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5969-3629-471f-9477-82945aa54842 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.498255 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.499108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.499433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.499774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] PATH_INFO: `/v3/projects/715d4ad98f674628a09d1ef51119a847/users/077e456dc22944f9ba1fd348f1990d58/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.518211 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=715d4ad98f674628a09d1ef51119a847, user_id=077e456dc22944f9ba1fd348f1990d58, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.521446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.537827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1391/2779] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:28 2022] PUT /identity/v3/projects/715d4ad98f674628a09d1ef51119a847/users/077e456dc22944f9ba1fd348f1990d58/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:28.542227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.554346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdccd5f-e4af-465e-a160-18484e1b1cb6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.606351 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:28.621905 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.622528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.622808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.623100 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.623743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.625372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ab78f-fec8-4ae1-8433-383c8e042270 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.637655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1389/2780] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:28.642007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.656706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896de30-cb88-465a-861a-75fc13f1c0ce admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.662471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.663149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.663421 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.663689 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] PATH_INFO: `/v3/projects/715d4ad98f674628a09d1ef51119a847/users/077e456dc22944f9ba1fd348f1990d58/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.665636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] RBAC: Authorizing `identity:create_grant(project_id=715d4ad98f674628a09d1ef51119a847, user_id=077e456dc22944f9ba1fd348f1990d58, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.668340 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.679953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1392/2781] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:28 2022] PUT /identity/v3/projects/715d4ad98f674628a09d1ef51119a847/users/077e456dc22944f9ba1fd348f1990d58/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:28.691168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.691559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.692378 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.723855 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] MFA Rules not processed for user `077e456dc22944f9ba1fd348f1990d58`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:28.725392 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:28.804841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1390/2782] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:28.816379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:28.818964 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:28.881736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca5dfa-9c05-4c36-9d55-7ea354cf458e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:28.885453 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:28.886094 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:28.886375 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:28.886639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:28.962355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:28.965526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:28.967775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:28.978562 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1393/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:30.514817 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:30.516390 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:30.528857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728542-9310-4b46-9df5-6fe27773a8a2 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:30.533107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:30.533806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:30.534092 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:30.534366 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:30.595592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:30.597224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:30.609568 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1391/2784] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:31.169612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:31.178934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3905e9-8cd7-401a-b44a-3b74f3d7ca2d service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:31.182578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:31.183177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:31.183452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:31.183717 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:31.193770 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:31.195130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa473f-9430-42cf-9a4f-0aaa1fd90fde service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:31.205007 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1394/2785] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:36.203457 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:36.278742 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc9b07-c813-4ff7-a7e0-8f37050071ea service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:36.283311 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:36.283941 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:36.284267 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:36.284566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] PATH_INFO: `/v3/users/da5b83f300c544bc956e2e1ad1fcc8bd` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:36.285740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5b83f300c544bc956e2e1ad1fcc8bd)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:36.287900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:36.293431 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=100 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:36.337027 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5b83f300c544bc956e2e1ad1fcc8bd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.337698 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5b83f300c544bc956e2e1ad1fcc8bd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.350168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5b83f300c544bc956e2e1ad1fcc8bd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.357297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5b83f300c544bc956e2e1ad1fcc8bd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.357862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5b83f300c544bc956e2e1ad1fcc8bd'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.371543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1392/2786] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:36 2022] DELETE /identity/v3/users/da5b83f300c544bc956e2e1ad1fcc8bd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:36.376376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:36.376882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:36.376981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:36.397309 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:36.398559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:36.461053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1395/2787] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:36.470910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:36.472359 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:36.535799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9321-59a7-46b4-9bd6-fbeddafc12a7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:36.539823 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:36.540440 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:36.540718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:36.540993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:36.551580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:36.553761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:36.565007 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1393/2788] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:30:36.857369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:36.860820 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:36.879416 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388529c5-45a9-439c-86fe-ada899dd62fa None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:36.887039 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:36.887768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:36.888051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:36.888354 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] PATH_INFO: `/v3/projects/cd16aee0d02e443fa2573def6a246ab4` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:36.889709 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd16aee0d02e443fa2573def6a246ab4)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:36.891616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:36.943267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd16aee0d02e443fa2573def6a246ab4', 'request_id': 'req-79d28f14-1516-48dc-a3b4-d9ebc575a32f'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.949612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd16aee0d02e443fa2573def6a246ab4', 'request_id': 'req-79d28f14-1516-48dc-a3b4-d9ebc575a32f'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.957174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd16aee0d02e443fa2573def6a246ab4', 'request_id': 'req-79d28f14-1516-48dc-a3b4-d9ebc575a32f'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.969044 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] The token cache is being invalidate because project cd16aee0d02e443fa2573def6a246ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:36.969349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.969983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:36.971456 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1396/2789] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:36 2022] DELETE /identity/v3/projects/cd16aee0d02e443fa2573def6a246ab4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:36.976188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:36.986544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107515e0-ce0e-48e0-81d2-ce9b2ac8e956 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:37.046881 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:37.047588 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:37.047867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:37.048179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] PATH_INFO: `/v3/users/709da4bf48a04f7fa2fcbd31ffa1d558` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:37.049417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=709da4bf48a04f7fa2fcbd31ffa1d558)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:37.051976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:37.089528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:37.108308 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709da4bf48a04f7fa2fcbd31ffa1d558'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.108965 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709da4bf48a04f7fa2fcbd31ffa1d558'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.121799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709da4bf48a04f7fa2fcbd31ffa1d558'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.129627 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709da4bf48a04f7fa2fcbd31ffa1d558'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.130211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709da4bf48a04f7fa2fcbd31ffa1d558'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.142775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1394/2790] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:36 2022] DELETE /identity/v3/users/709da4bf48a04f7fa2fcbd31ffa1d558 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:37.430970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:37.432489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:37.446315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d28f14-1516-48dc-a3b4-d9ebc575a32f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:37.506406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:37.507047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:37.507320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:37.507591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] PATH_INFO: `/v3/projects/2712f86c0b34449cb3bc8fa0647325f5` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:37.508724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2712f86c0b34449cb3bc8fa0647325f5)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:37.510674 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:37.518754 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:37.562207 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2712f86c0b34449cb3bc8fa0647325f5', 'request_id': 'req-41ae83a2-52e7-4026-8054-54ee83407a09'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.567694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2712f86c0b34449cb3bc8fa0647325f5', 'request_id': 'req-41ae83a2-52e7-4026-8054-54ee83407a09'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.576104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2712f86c0b34449cb3bc8fa0647325f5', 'request_id': 'req-41ae83a2-52e7-4026-8054-54ee83407a09'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.588196 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] The token cache is being invalidate because project 2712f86c0b34449cb3bc8fa0647325f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:37.588487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.589220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.591294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1397/2791] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:37 2022] DELETE /identity/v3/projects/2712f86c0b34449cb3bc8fa0647325f5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:37.595874 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:37.598077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:37.612319 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f09c8-a764-4703-9940-7c1f84026e96 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:37.798701 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:37.799323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:37.799598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:37.799867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] PATH_INFO: `/v3/users/077e456dc22944f9ba1fd348f1990d58` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:37.800986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] RBAC: Authorizing `identity:delete_user(user_id=077e456dc22944f9ba1fd348f1990d58)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:37.803168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:37.850380 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077e456dc22944f9ba1fd348f1990d58'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.851066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077e456dc22944f9ba1fd348f1990d58'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.863298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077e456dc22944f9ba1fd348f1990d58'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.871717 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077e456dc22944f9ba1fd348f1990d58'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.872354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3c8bea22-3e8b-4398-9381-3fe0092f319c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077e456dc22944f9ba1fd348f1990d58'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:37.886560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1395/2792] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:37 2022] DELETE /identity/v3/users/077e456dc22944f9ba1fd348f1990d58 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:30:38.157953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.159379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:38.172265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae83a2-52e7-4026-8054-54ee83407a09 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.238463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.239016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.239298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.239567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] PATH_INFO: `/v3/projects/715d4ad98f674628a09d1ef51119a847` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.241303 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=715d4ad98f674628a09d1ef51119a847)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.243266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.293400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715d4ad98f674628a09d1ef51119a847', 'request_id': 'req-6cf9e687-8b85-487e-b67f-b7ff603df909'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:38.299158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715d4ad98f674628a09d1ef51119a847', 'request_id': 'req-6cf9e687-8b85-487e-b67f-b7ff603df909'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:38.307113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715d4ad98f674628a09d1ef51119a847', 'request_id': 'req-6cf9e687-8b85-487e-b67f-b7ff603df909'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:38.318559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] The token cache is being invalidate because project 715d4ad98f674628a09d1ef51119a847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:38.318961 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:38.319609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-6cf9e687-8b85-487e-b67f-b7ff603df909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:38.321101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1398/2793] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:38 2022] DELETE /identity/v3/projects/715d4ad98f674628a09d1ef51119a847 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:38.325916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.326203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.326475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.345765 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:38.346982 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:38.368703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:38.408894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1396/2794] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:38.417409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.417721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.417994 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.438634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:38.439865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:38.452601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1399/2795] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:38.456877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.467245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede48add-f82a-4ce7-bee5-2f6c3790d210 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.471801 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.472389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.472664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.472933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.473573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:30:38.473907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.476609 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.488041 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1397/2796] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:30:38.493000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.502613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca485b-2ab6-4a35-b5cd-2d92a7e7e68c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.507404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.508013 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.508289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.508555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.509215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.511124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.534792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:38.543087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1400/2797] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:30:38 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:38.547130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.559453 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fb427-9afe-42ad-ae7d-ff1e57a24fe7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.616575 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.617211 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.617439 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.617737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.618550 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.620467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.643056 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1398/2798] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:30:38 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:30:38.648028 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.659763 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46299e-863e-4dd8-9c93-9d912de132e8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.672849 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.673981 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.674458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.674927 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.676104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.679655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.695342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1401/2799] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:38.699533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.710186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f5ac-5376-446f-8e0a-4fd6006bafa6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.715002 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.715598 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.715872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.716136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.716771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.718394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ac992-9a97-42ae-b626-6a73e8cc1e1c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.729190 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1399/2800] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:30:38.733350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.744075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b6ef-226c-41be-8310-b5445fffc288 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.748970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.749624 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.749956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.750251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] PATH_INFO: `/v3/projects/fb40fdfddeeb4f6caaa7f40deaaa08e1/users/9565f814642a4c29bf53cf7ed50987ad/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.766182 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb40fdfddeeb4f6caaa7f40deaaa08e1, user_id=9565f814642a4c29bf53cf7ed50987ad, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:38.768831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:38.783350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1402/2801] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:30:38 2022] PUT /identity/v3/projects/fb40fdfddeeb4f6caaa7f40deaaa08e1/users/9565f814642a4c29bf53cf7ed50987ad/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:38.795664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.795948 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.796220 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:38.834419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] MFA Rules not processed for user `9565f814642a4c29bf53cf7ed50987ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:38.835927 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:38.915257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1400/2802] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:38.927128 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:38.928997 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:38.988991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59002039-86b4-45a2-96fc-d48d23fbb35b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:38.992680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:38.993281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:38.993557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:38.993869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:39.002342 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:39.003691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbf7cd-88d9-441b-b591-d65dbb886ba5 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:39.013669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1403/2803] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:42.630640 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:42.656360 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:42.695335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f4eed-3736-4533-9094-c48b76743f4d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:42.699854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:42.700468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:42.700744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:42.701016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] PATH_INFO: `/v3/users/9565f814642a4c29bf53cf7ed50987ad` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:42.702184 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9565f814642a4c29bf53cf7ed50987ad)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:42.704126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:42.750679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9565f814642a4c29bf53cf7ed50987ad'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:42.751318 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9565f814642a4c29bf53cf7ed50987ad'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:42.763113 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9565f814642a4c29bf53cf7ed50987ad'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:42.770346 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9565f814642a4c29bf53cf7ed50987ad'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:42.771463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9565f814642a4c29bf53cf7ed50987ad'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:42.784252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1401/2804] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:30:42 2022] DELETE /identity/v3/users/9565f814642a4c29bf53cf7ed50987ad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:42.789361 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:42.789906 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:42.790015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:42.808750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:30:42.809970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:42.866521 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1404/2805] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:30:42.883096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:42.886614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:42.946509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1d12-78b9-43f1-b930-28f20e494a67 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:42.950877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:42.951488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:42.951760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:42.952028 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:42.961569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:42.963057 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:42.972614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1402/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:30:43.251468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:30:43.252986 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:30:43.270022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf2ad6-95a5-43d6-80ff-3370866a941c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:30:43.276251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:30:43.277102 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:30:43.277775 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:30:43.278110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] PATH_INFO: `/v3/projects/fb40fdfddeeb4f6caaa7f40deaaa08e1` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:30:43.279293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb40fdfddeeb4f6caaa7f40deaaa08e1)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:30:43.281216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:30:43.329573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb40fdfddeeb4f6caaa7f40deaaa08e1', 'request_id': 'req-58e50e11-b4dd-4e4e-a865-311e4665bc72'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:43.336701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb40fdfddeeb4f6caaa7f40deaaa08e1', 'request_id': 'req-58e50e11-b4dd-4e4e-a865-311e4665bc72'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:43.339296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:43.344532 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb40fdfddeeb4f6caaa7f40deaaa08e1', 'request_id': 'req-58e50e11-b4dd-4e4e-a865-311e4665bc72'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:43.358321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] The token cache is being invalidate because project fb40fdfddeeb4f6caaa7f40deaaa08e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:30:43.358321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:43.358892 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-58e50e11-b4dd-4e4e-a865-311e4665bc72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:30:43.360369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1405/2807] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:30:43 2022] DELETE /identity/v3/projects/fb40fdfddeeb4f6caaa7f40deaaa08e1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:30:52.968389 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:30:53.354973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=1,keystone:SELECT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:07.390686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:07.392139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:07.405778 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2038270-55ad-41b2-bc71-7fd2f3b31df3 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:07.461540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:07.462187 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:07.462459 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:07.462721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] PATH_INFO: `/v3/users/8e6a8778a7f44ace9927ca6ec474158d` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:07.463846 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6a8778a7f44ace9927ca6ec474158d)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:07.465769 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:07.512089 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6a8778a7f44ace9927ca6ec474158d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:07.512747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6a8778a7f44ace9927ca6ec474158d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:07.525114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6a8778a7f44ace9927ca6ec474158d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:07.532764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6a8778a7f44ace9927ca6ec474158d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:07.533379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6a8778a7f44ace9927ca6ec474158d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:07.545533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1403/2808] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:31:07 2022] DELETE /identity/v3/users/8e6a8778a7f44ace9927ca6ec474158d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:07.550071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:07.550578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:07.550692 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:07.581086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:07.582394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:07.645879 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1406/2809] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:07.656075 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:07.657512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:07.715344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19de28-5d97-4074-a1bc-64efdbd6d1e6 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:07.719030 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:07.719629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:07.719907 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:07.720179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:07.729281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:07.730738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001810d-bda9-40d9-8e08-775ce709af2a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:07.741001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1404/2810] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:31:08.011215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.012790 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:08.025283 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a26da-c216-4f17-854e-dfbaac6583db None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.030900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.031573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.031864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.032114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] PATH_INFO: `/v3/projects/47ee5e0c9d7b45f8b2e10a338c20ea01` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.033250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ee5e0c9d7b45f8b2e10a338c20ea01)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.035282 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.084174 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ee5e0c9d7b45f8b2e10a338c20ea01', 'request_id': 'req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:08.089602 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ee5e0c9d7b45f8b2e10a338c20ea01', 'request_id': 'req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:08.096928 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ee5e0c9d7b45f8b2e10a338c20ea01', 'request_id': 'req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:08.108942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] The token cache is being invalidate because project 47ee5e0c9d7b45f8b2e10a338c20ea01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:31:08.109232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:08.109904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-d90c3f27-bf12-4003-8360-1cc1e4ab2c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:08.111408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1407/2811] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:31:08 2022] DELETE /identity/v3/projects/47ee5e0c9d7b45f8b2e10a338c20ea01 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:08.116223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.116506 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.116775 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.137610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:08.138860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:08.156423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:08.208468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1405/2812] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:08.217320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.217727 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.218093 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.240267 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:08.241792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:08.257900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1408/2813] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:08.261939 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.271985 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f727-c073-466a-ac0e-c3e8a8b2548d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.277864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.278472 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.278745 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.279254 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.280018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:31:08.280018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.281546 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.294658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1406/2814] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 12:31:08.298989 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.309431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc507c39-c323-444f-ba2a-8c1957574fb2 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.314398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.315033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.315319 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.315682 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.316278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.318260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.347216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1409/2815] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:31:08 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:31:08.352471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.362805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6de9-1376-4aed-aba9-8a0ff709f6d0 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.428691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.429419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.429585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.429894 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.430527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.432442 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.454106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1407/2816] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:31:08 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:31:08.458221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.469505 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187356fd-a487-4dea-8126-bdf5fb265eba admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.474409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.475014 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.475302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.475573 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.476227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.478484 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.489618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1410/2817] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:31:08.493808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.504015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dbc70-723e-48f3-8c05-ec5a731ff273 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.508703 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.509299 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.509573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.509884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.510527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.512139 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dccee-fe9f-431c-a8ec-9fac7e5a0ed3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.522759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1408/2818] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:31:08.527708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.538698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ad3ea-9cb9-491e-89b6-44df2857be80 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.544839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.545604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.546009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.546389 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] PATH_INFO: `/v3/projects/4f5ce305762c463ea30de155dab4a7c7/users/b3fe688e748343b598a90a8687011952/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.561979 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5ce305762c463ea30de155dab4a7c7, user_id=b3fe688e748343b598a90a8687011952, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.564835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.578750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1411/2819] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:31:08 2022] PUT /identity/v3/projects/4f5ce305762c463ea30de155dab4a7c7/users/b3fe688e748343b598a90a8687011952/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:08.591612 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.591898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.592175 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.621084 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] MFA Rules not processed for user `b3fe688e748343b598a90a8687011952`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:08.622253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:08.691655 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1409/2820] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:08.704112 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:08.705907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:08.730064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:08.787802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a275af6-7162-48bd-9732-9c8396234cd2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:08.791535 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:08.792132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:08.792404 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:08.792670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:08.800837 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:08.802276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390856c-d189-4804-8b6e-25acf9eee2a9 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:08.811276 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1412/2821] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:31:18.688573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:18.807580 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=26 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:57.949491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.020310 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61fd5e-4c1e-46f6-a3a1-e9a4e8dc7d24 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:58.025292 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:58.026016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.026296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.026572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] PATH_INFO: `/v3/users/b3fe688e748343b598a90a8687011952` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.029145 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fe688e748343b598a90a8687011952)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:58.031158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:58.079658 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fe688e748343b598a90a8687011952'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.080324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fe688e748343b598a90a8687011952'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.093536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fe688e748343b598a90a8687011952'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.102324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fe688e748343b598a90a8687011952'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.102943 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fe688e748343b598a90a8687011952'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.118541 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1410/2822] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:31:57 2022] DELETE /identity/v3/users/b3fe688e748343b598a90a8687011952 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:58.122863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.123380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.123490 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.144411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:58.145629 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:58.206491 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1413/2823] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:58.217767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.220313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:58.301221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294a0a-7517-43b9-a780-5bb4f197d31c admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:58.305605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:58.306239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.306514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.306781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.316154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:58.317585 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932bbd-ff89-4a8d-b101-30e3c70a1248 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:58.327312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1411/2824] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:31:58.598485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.600038 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:58.613353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f501-85d6-4e93-9ced-0d6bebca0285 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:58.618194 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:58.618835 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.619109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.619381 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] PATH_INFO: `/v3/projects/4f5ce305762c463ea30de155dab4a7c7` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.620539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5ce305762c463ea30de155dab4a7c7)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:58.623162 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:58.677428 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5ce305762c463ea30de155dab4a7c7', 'request_id': 'req-8ab45679-6b6f-4190-8165-308e4081e57d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.682919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5ce305762c463ea30de155dab4a7c7', 'request_id': 'req-8ab45679-6b6f-4190-8165-308e4081e57d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.690962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5ce305762c463ea30de155dab4a7c7', 'request_id': 'req-8ab45679-6b6f-4190-8165-308e4081e57d'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.702634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] The token cache is being invalidate because project 4f5ce305762c463ea30de155dab4a7c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:31:58.702925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.703601 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-8ab45679-6b6f-4190-8165-308e4081e57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:31:58.705103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1414/2825] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:31:58 2022] DELETE /identity/v3/projects/4f5ce305762c463ea30de155dab4a7c7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:58.710095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.710381 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.710654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.718320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:58.733951 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:58.735767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:58.795376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1412/2826] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:58.803294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.803571 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.803861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.823729 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:58.824943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:58.833467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1415/2827] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:58.837764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.848296 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601fefb-5c0d-4a51-a445-09593708119c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:58.853708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:58.853893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.854172 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.854444 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.855480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:31:58.855480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:58.858102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:58.870284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1413/2828] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:31:58.875776 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.887181 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654545-cec8-454c-b8b7-34f2db74527c None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:58.892865 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:58.893488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:58.893799 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:58.894065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:58.894701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:58.896582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:58.929687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1416/2829] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:31:58 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:31:58.934037 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:58.944886 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370230-2090-4e37-a81e-df6daddb1461 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:59.010653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:59.011382 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.011532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.011811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.012475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:59.014454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:59.038018 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1414/2830] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:31:58 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:31:59.044543 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:59.055630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f16e9-0352-458e-a0e9-d1de62209735 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:59.060458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:59.061070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.061346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.061607 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.062299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:59.063915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:59.074780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1417/2831] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:31:59.078942 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:59.089664 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c833f9f-6840-4782-92ea-dd54362df4dd admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:59.095343 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:59.095957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.096227 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.096494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.097234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:59.098781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5b9ce-8d77-44d3-ad87-e29f6eec8b13 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:59.110437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1415/2832] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:31:59.114633 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:59.125131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74209a3b-996e-480b-ab94-a69028a7180f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:59.130287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:59.130968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.131281 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.131556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] PATH_INFO: `/v3/projects/4e13494666564124bdf4007277a515d6/users/1102506cce6d4bd68d1af92fa8811fe3/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.149183 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e13494666564124bdf4007277a515d6, user_id=1102506cce6d4bd68d1af92fa8811fe3, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:59.151597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:59.171485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1418/2833] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:31:59 2022] PUT /identity/v3/projects/4e13494666564124bdf4007277a515d6/users/1102506cce6d4bd68d1af92fa8811fe3/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:31:59.177726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e178504-fd12-47e9-84a4-e360c49f2baf None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.178216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e178504-fd12-47e9-84a4-e360c49f2baf None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.178689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e178504-fd12-47e9-84a4-e360c49f2baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.199118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2e178504-fd12-47e9-84a4-e360c49f2baf None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:31:59.201325 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2e178504-fd12-47e9-84a4-e360c49f2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:59.286999 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1416/2834] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:31:59.297057 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:31:59.298805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:31:59.322394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:31:59.363357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d917b3-3a3b-4d9c-9dec-203c55c7b682 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:31:59.367222 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:31:59.367829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:31:59.368100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:31:59.368362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:31:59.378086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:31:59.379540 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:31:59.390235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1419/2835] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:03.404345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.404932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.404932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.426460 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:03.439919 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] MFA Rules not processed for user `1102506cce6d4bd68d1af92fa8811fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:32:03.442680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:03.535099 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1417/2836] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:32:03.540464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.551336 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63038-65dd-42fc-b9f9-984c89d631d2 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.556348 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.556982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.557256 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.557529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.558212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.560750 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.590616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1420/2837] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:32:03.594860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.607680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e996-3af8-40a4-9815-fc19679cf75f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.673528 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.674203 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.674477 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.674774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.675428 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.677430 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.704870 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1418/2838] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:32:03 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:32:03.709015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.719894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04d6b2-a725-420d-82c8-acab52446842 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.728221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.728221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.728221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.728221 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.728969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.728969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.742020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1421/2839] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:32:03.747560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.763282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6511a-8318-403e-a9a2-0d6f5fe097bb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.769523 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.770312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.771146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.771277 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] PATH_INFO: `/v3/projects/69d07392210b4dca8703e02c4e09153a/users/ee4674549863473c9edabc6bf941982b/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.789182 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d07392210b4dca8703e02c4e09153a, user_id=ee4674549863473c9edabc6bf941982b, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.791896 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.807980 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1419/2840] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:32:03 2022] PUT /identity/v3/projects/69d07392210b4dca8703e02c4e09153a/users/ee4674549863473c9edabc6bf941982b/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:32:03.813103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.824482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ea017-1392-4d00-8949-2c758d70ac8d admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.894191 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.894842 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.895131 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.895419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.896076 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.897781 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.909813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1422/2841] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:32:03.914205 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:03.927293 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1eb7-0d70-4b52-b6c0-da753a89d290 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:03.932394 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:03.933070 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:03.933352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:03.933711 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] PATH_INFO: `/v3/projects/69d07392210b4dca8703e02c4e09153a/users/ee4674549863473c9edabc6bf941982b/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:03.936376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d07392210b4dca8703e02c4e09153a, user_id=ee4674549863473c9edabc6bf941982b, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:03.939140 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba893b3-4ecd-4e65-a1f6-f63a33d050e9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:03.951256 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1420/2842] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:32:03 2022] PUT /identity/v3/projects/69d07392210b4dca8703e02c4e09153a/users/ee4674549863473c9edabc6bf941982b/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:32:04.510513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:04.512347 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:04.606542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9755c-a4cc-4145-be69-d69ad8968888 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:04.674226 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:04.687884 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:04.689210 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:04.690789 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:04.691386 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:04.694866 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:04.849873 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:04.851694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:04.881462 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1423/2843] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:08.830043 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddabc6-7140-4f89-b602-6bb361d363ee None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:08.830530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddabc6-7140-4f89-b602-6bb361d363ee None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:08.830650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddabc6-7140-4f89-b602-6bb361d363ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:08.872106 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-2eddabc6-7140-4f89-b602-6bb361d363ee None None] MFA Rules not processed for user `ee4674549863473c9edabc6bf941982b`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:32:08.873446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-2eddabc6-7140-4f89-b602-6bb361d363ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:08.932810 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:08.985553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1421/2844] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:32:09.046807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:09.050514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:09.173541 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef1e29-6378-4da0-865d-38bf16802c06 service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:09.179679 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:09.180488 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:09.180741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:09.181042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:09.236948 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=100 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:09.257152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:09.266567 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:09.277741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1424/2845] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:10.355723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:10.355723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:10.355723 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:10.379059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] MFA Rules not processed for user `1102506cce6d4bd68d1af92fa8811fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:32:10.380558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:10.388805 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1422/2846] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:32:19.272667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:20.369253 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=14 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:32.335118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:32.336577 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:32.393868 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28332b-3d0d-457b-b7be-9f36da99c700 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:32.397396 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:32.398027 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:32.398301 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:32.398565 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:32.400288 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:32.410945 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:32.412349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:32.420346 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1425/2847] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:32:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:35.024221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:35.034873 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168ca7b-e7ea-461c-8bed-ce8a21908eeb None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:35.038619 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:35.039224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:35.039718 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:35.040188 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:35.042370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:32:35.053681 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:35.055142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:35.065762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1423/2848] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:42.416719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=26 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:45.060073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:32:49.969974 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:32:49.978547 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0ecc9-e396-4c15-9348-60403fca9163 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:32:49.982394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:32:49.983092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:32:49.983373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:32:49.983649 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:32:49.993056 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:32:49.994525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:32:50.004774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1426/2849] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:32:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:32:59.999785 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=6 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:05.550282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:05.559001 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb46c8f-1e31-446c-b3ae-1062bb3d55c5 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:05.562596 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:05.563272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:05.563551 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:05.563956 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:05.565733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:05.628845 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:05.630622 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:05.640710 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1424/2850] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:09.032858 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.035791 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:09.105240 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.107087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:09.122081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8565a-22c8-4080-87fe-60b324e4c646 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.131152 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748fb1-4695-47d1-a344-f0b37b6f989e service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.133441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.134119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.134395 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.134667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] PATH_INFO: `/v3/users/e235718ad1da42ebb209fb4b13626ff1` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.135797 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e235718ad1da42ebb209fb4b13626ff1)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.136409 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.137021 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.137298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.137566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.137923 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.139639 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:09.152921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.154554 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.165719 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1427/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:09.191949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e235718ad1da42ebb209fb4b13626ff1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.192610 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e235718ad1da42ebb209fb4b13626ff1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.208180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e235718ad1da42ebb209fb4b13626ff1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.215251 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e235718ad1da42ebb209fb4b13626ff1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.215903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0122f99f-4e64-457c-a24b-7b32f0591df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e235718ad1da42ebb209fb4b13626ff1'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.229665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1425/2852] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:09 2022] DELETE /identity/v3/users/e235718ad1da42ebb209fb4b13626ff1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:09.414105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.426675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772b2e-40df-4f55-ba29-e28a921647f6 service placement] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.486408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.487073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.487339 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.487613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] PATH_INFO: `/v3/projects/a896d72b5b00477d855131345711c748` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.488784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a896d72b5b00477d855131345711c748)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.490739 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.538477 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a896d72b5b00477d855131345711c748', 'request_id': 'req-db762a1a-5f80-4915-98e3-6eec540d097c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.544406 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a896d72b5b00477d855131345711c748', 'request_id': 'req-db762a1a-5f80-4915-98e3-6eec540d097c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.552272 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a896d72b5b00477d855131345711c748', 'request_id': 'req-db762a1a-5f80-4915-98e3-6eec540d097c'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.563788 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] The token cache is being invalidate because project a896d72b5b00477d855131345711c748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:09.564077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.564726 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-db762a1a-5f80-4915-98e3-6eec540d097c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:09.566271 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1428/2853] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:09 2022] DELETE /identity/v3/projects/a896d72b5b00477d855131345711c748 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:09.572186 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.572469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.572743 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.593435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:09.595675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:09.665586 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1426/2854] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:09.673411 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.673720 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.673993 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.693466 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:09.694699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:09.702503 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1429/2855] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:09.707180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.716875 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1c742-ec38-40b6-b077-dbd54f17c061 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.722560 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.723284 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.723558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.723825 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.724466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:09.724793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.726433 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.728010 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:09.740213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1427/2856] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:33:09.744529 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.754059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61738a0-32ea-4632-8c4c-22d0a466d9a5 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.758780 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.759400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.759672 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.759935 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.760553 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.762476 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.774065 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:09.792688 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1430/2857] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:09 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:09.796819 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.806876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ae66-1a8a-46ae-bf47-6b95d921d204 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.862983 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.863591 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.863867 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.864137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.864781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.866735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.888354 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1428/2858] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:09 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:33:09.893605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.904206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0126f6-294a-4f97-a883-41ccbe803832 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.910258 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.910862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.911140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.911407 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.912080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.913700 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.924728 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1431/2859] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:09.928792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.939734 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc0783-379a-420b-ab2d-d33a68bb2a3b admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.945078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.945712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.946023 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.946298 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.946955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.948566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5843234-1f43-46f1-b6f5-287a4808fa9e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:09.960492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1429/2860] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:09.964706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:09.975691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12eae6-83ce-4b8e-951d-cfd7558ee4d3 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:09.980383 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:09.981032 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:09.981311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:09.981586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] PATH_INFO: `/v3/projects/3a5d42a3868944e38d3439b6b117ecad/users/4e1b84022d544fe08f4e85596c195ba4/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:09.996587 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5d42a3868944e38d3439b6b117ecad, user_id=4e1b84022d544fe08f4e85596c195ba4, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:09.999285 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:10.013924 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1432/2861] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:09 2022] PUT /identity/v3/projects/3a5d42a3868944e38d3439b6b117ecad/users/4e1b84022d544fe08f4e85596c195ba4/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:10.025156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:10.025436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:10.025740 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:10.053827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] MFA Rules not processed for user `4e1b84022d544fe08f4e85596c195ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:10.055032 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:10.117957 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1430/2862] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:10.133216 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:10.135051 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:10.222881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7d30-ec97-4818-816c-0d78baaf5602 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:10.229110 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:10.229778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:10.230053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:10.230374 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:10.242584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:10.244206 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37702930-1ed9-41d0-ad44-9d9058b795ce service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:10.256686 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1433/2863] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:19.676852 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:19.740192 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7aa543-2711-4d08-ad2d-faee415666fe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:19.744912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:19.745423 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:19.745721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:19.745993 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] PATH_INFO: `/v3/users/4e1b84022d544fe08f4e85596c195ba4` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:19.747116 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1b84022d544fe08f4e85596c195ba4)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:19.749048 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:19.751626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:19.801975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1b84022d544fe08f4e85596c195ba4'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:19.802679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1b84022d544fe08f4e85596c195ba4'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:19.815050 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1b84022d544fe08f4e85596c195ba4'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:19.822595 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1b84022d544fe08f4e85596c195ba4'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:19.823333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1b84022d544fe08f4e85596c195ba4'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:19.835714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1431/2864] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:19 2022] DELETE /identity/v3/users/4e1b84022d544fe08f4e85596c195ba4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:19.840380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:19.840875 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:19.840980 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:19.860941 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:19.862798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:19.923279 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1434/2865] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:19.933761 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:19.935194 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:19.992162 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c5e14-35c2-4666-a5a3-bcb4f6cf0eb2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:19.995911 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:19.996512 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:19.996790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:19.997063 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.006587 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.008029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33640cff-437f-4c84-b158-6a530700e12a service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.017488 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1432/2866] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:20.295100 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.296680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:20.311542 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6175c6-52d9-451a-a66d-4242fb411550 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.318139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.318963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.319240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.319591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] PATH_INFO: `/v3/projects/3a5d42a3868944e38d3439b6b117ecad` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.321140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5d42a3868944e38d3439b6b117ecad)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.323880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.370931 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:20.381733 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5d42a3868944e38d3439b6b117ecad', 'request_id': 'req-9843a24e-df34-4596-bd41-a644399c4026'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:20.387349 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5d42a3868944e38d3439b6b117ecad', 'request_id': 'req-9843a24e-df34-4596-bd41-a644399c4026'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:20.394963 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5d42a3868944e38d3439b6b117ecad', 'request_id': 'req-9843a24e-df34-4596-bd41-a644399c4026'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:20.409251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] The token cache is being invalidate because project 3a5d42a3868944e38d3439b6b117ecad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:20.409886 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:20.410804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-9843a24e-df34-4596-bd41-a644399c4026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:20.412310 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1435/2867] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:20 2022] DELETE /identity/v3/projects/3a5d42a3868944e38d3439b6b117ecad => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:20.417323 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.417601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.417976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.439695 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:20.441135 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:20.503518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1433/2868] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:20.512394 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.512721 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.513067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.536871 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:20.538326 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:20.547559 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1436/2869] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:20.552841 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.563138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4374-c8f9-42ad-a17d-69c2a1535b3c None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.567917 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.568522 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.568799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.569096 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.569781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:20.570087 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.571731 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.583733 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1434/2870] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:33:20.588311 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.598291 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00960799-6200-47c1-ad4f-96310ac906a9 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.602991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.603597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.603869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.604135 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.604773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.606703 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.636940 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1437/2871] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:20.641239 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.651949 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524b72-215e-4f0d-ab99-159fba421a7f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.662097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:20.715784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.716576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.716685 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.716902 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.717572 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.719511 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.742483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1435/2872] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:33:20.746756 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.758475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e937b1a-73a2-418e-bde5-fe8d3a3c3e6e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.763329 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.764006 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.764300 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.764551 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.765204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.766852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.777713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1438/2873] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:20.782072 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.792901 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7329-c23e-41d4-8236-3be4b9169b29 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.797605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.798241 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.798518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.798788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.799614 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.801263 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26086-22d2-42e6-ac46-cb2a66216341 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.812447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1436/2874] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:20.816860 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:20.828438 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036627e-76a9-42fd-8d39-e1a99d7c58fb admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:20.834163 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:20.834844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.835158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.835443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] PATH_INFO: `/v3/projects/950d3f5a7e044b91bdf558345019c6e2/users/ea995b22ebe24458b56bbb3f53d1d241/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.852369 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] RBAC: Authorizing `identity:create_grant(project_id=950d3f5a7e044b91bdf558345019c6e2, user_id=ea995b22ebe24458b56bbb3f53d1d241, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:20.855113 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:20.869516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1439/2875] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:20 2022] PUT /identity/v3/projects/950d3f5a7e044b91bdf558345019c6e2/users/ea995b22ebe24458b56bbb3f53d1d241/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:20.879752 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:20.880033 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:20.880307 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:20.911209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] MFA Rules not processed for user `ea995b22ebe24458b56bbb3f53d1d241`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:20.912449 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:20.979404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1437/2876] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:20.984203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.047774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cb57d-7d58-4015-bedf-50bc79bb329f admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.052765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.053379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.053675 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.053943 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.054583 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.056471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.085198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1440/2877] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:20 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 12:33:21.089808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.103115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550caa9-b07c-48f8-8fb1-304bf8f0f0e0 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.168475 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.168998 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.169324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.169580 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.170235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.172413 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.195492 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1438/2878] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:21 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:21.199694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.214379 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133303c-b003-4fe5-87a6-8f9206e2db1c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.219290 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.219951 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.220274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.220586 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.221284 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.223115 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.234460 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1441/2879] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:21.238975 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.253984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0d70c-9142-4969-8ba8-fa53cb40e723 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.261474 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.262643 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.263148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.263649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] PATH_INFO: `/v3/projects/afb9483d85df49439c30cfe905767707/users/90cebe716e0941e1aa06df60a1423180/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.269849 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:21.289278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb9483d85df49439c30cfe905767707, user_id=90cebe716e0941e1aa06df60a1423180, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.292042 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.306483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1439/2880] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:21 2022] PUT /identity/v3/projects/afb9483d85df49439c30cfe905767707/users/90cebe716e0941e1aa06df60a1423180/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:21.310669 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.321933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e170ea7-03be-4a14-a5f1-ddcd32068ecf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.359059 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:21.387696 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.388308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.388589 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.388862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.389514 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.391149 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379b130-1d06-43a8-87b4-b4190a4e6a0f admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.402470 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1442/2881] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:21.407317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.419562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745425-6aae-444f-bd01-f62b7d6af444 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.425114 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.425947 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.426173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.426450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] PATH_INFO: `/v3/projects/afb9483d85df49439c30cfe905767707/users/90cebe716e0941e1aa06df60a1423180/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.428432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb9483d85df49439c30cfe905767707, user_id=90cebe716e0941e1aa06df60a1423180, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.431150 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.442663 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1440/2882] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:21 2022] PUT /identity/v3/projects/afb9483d85df49439c30cfe905767707/users/90cebe716e0941e1aa06df60a1423180/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:21.453302 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.453591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.453904 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.484784 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] MFA Rules not processed for user `90cebe716e0941e1aa06df60a1423180`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:21.486042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:21.555299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1443/2883] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:21.566808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:21.568544 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:21.625903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89641a3-035e-40d4-829f-fc4c961e62d7 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:21.629720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:21.630328 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:21.630604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:21.630877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:21.640300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:21.641771 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c02497-4afa-4b64-b382-a06a4c52cc2a service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:21.651916 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1441/2884] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:22.046662 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:22.112017 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1944a-9c88-4c85-b606-a0ee70736203 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:22.117035 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:22.117712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:22.118010 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:22.118264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] PATH_INFO: `/v3/users/ea995b22ebe24458b56bbb3f53d1d241` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:22.119412 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea995b22ebe24458b56bbb3f53d1d241)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:22.121333 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:22.171673 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea995b22ebe24458b56bbb3f53d1d241'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.178777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea995b22ebe24458b56bbb3f53d1d241'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.183604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:22.194280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea995b22ebe24458b56bbb3f53d1d241'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.195298 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea995b22ebe24458b56bbb3f53d1d241'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.202332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea995b22ebe24458b56bbb3f53d1d241'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.209422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1444/2885] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:22 2022] DELETE /identity/v3/users/ea995b22ebe24458b56bbb3f53d1d241 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:22.213996 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:22.214483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:22.214599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:22.234618 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:22.235750 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:22.302712 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1442/2886] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:22.312661 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:22.314124 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:22.375883 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665448-2299-4cb0-9d90-ff0e6ea0db19 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:22.381575 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:22.382202 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:22.382474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:22.382760 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:22.392794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:22.395020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:22.404864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1445/2887] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:22.676509 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:22.678062 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:22.690650 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349b09d-877a-4432-abcd-6c9ae6cddd4a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:22.695851 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:22.696468 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:22.696751 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:22.697024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] PATH_INFO: `/v3/projects/950d3f5a7e044b91bdf558345019c6e2` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:22.698216 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=950d3f5a7e044b91bdf558345019c6e2)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:22.700377 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:22.749524 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950d3f5a7e044b91bdf558345019c6e2', 'request_id': 'req-b32645fd-34c3-475c-874b-de8f84ad0c7a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.757146 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950d3f5a7e044b91bdf558345019c6e2', 'request_id': 'req-b32645fd-34c3-475c-874b-de8f84ad0c7a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.762762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950d3f5a7e044b91bdf558345019c6e2', 'request_id': 'req-b32645fd-34c3-475c-874b-de8f84ad0c7a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.770102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:22.775266 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] The token cache is being invalidate because project 950d3f5a7e044b91bdf558345019c6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:22.775781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.776824 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.778735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1443/2888] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:22 2022] DELETE /identity/v3/projects/950d3f5a7e044b91bdf558345019c6e2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:22.782773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:22.792932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fa955-7557-4d14-b114-385cf39964f2 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:22.851367 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:22.852018 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:22.852293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:22.852557 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] PATH_INFO: `/v3/users/90cebe716e0941e1aa06df60a1423180` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:22.853711 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cebe716e0941e1aa06df60a1423180)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:22.855944 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:22.902600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cebe716e0941e1aa06df60a1423180'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.910734 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cebe716e0941e1aa06df60a1423180'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.922582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cebe716e0941e1aa06df60a1423180'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.923229 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cebe716e0941e1aa06df60a1423180'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.930278 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e5bad9c4-c5a9-4818-ba1b-b33017b70db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cebe716e0941e1aa06df60a1423180'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:22.931708 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:22.937509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1446/2889] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:22 2022] DELETE /identity/v3/users/90cebe716e0941e1aa06df60a1423180 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:23.212803 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.214289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:23.229170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32645fd-34c3-475c-874b-de8f84ad0c7a admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.291808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.292361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.292637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.292910 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] PATH_INFO: `/v3/projects/afb9483d85df49439c30cfe905767707` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.294115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb9483d85df49439c30cfe905767707)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.296101 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.347009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb9483d85df49439c30cfe905767707', 'request_id': 'req-ea83a5b9-603a-42a8-8743-e370a5b19f1f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:23.357340 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb9483d85df49439c30cfe905767707', 'request_id': 'req-ea83a5b9-603a-42a8-8743-e370a5b19f1f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:23.365111 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb9483d85df49439c30cfe905767707', 'request_id': 'req-ea83a5b9-603a-42a8-8743-e370a5b19f1f'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:23.376720 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] The token cache is being invalidate because project afb9483d85df49439c30cfe905767707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:23.376992 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:23.377691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-ea83a5b9-603a-42a8-8743-e370a5b19f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:23.379900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1444/2890] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:23 2022] DELETE /identity/v3/projects/afb9483d85df49439c30cfe905767707 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:23.386053 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.386768 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.387082 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.416487 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:23.417792 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:23.489518 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1447/2891] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:23.499288 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.499558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.499827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.523369 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:23.524963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:23.533567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1445/2892] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:23.538894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.549151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d55b4-9e30-4b9c-b5cf-bddb92d08553 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.554697 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.555332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.555612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.555887 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.556552 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87997) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:23.556857 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.560315 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.573695 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1448/2893] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:33:23.578367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.588317 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394dc9c-63a9-4769-9e4c-c669125348c7 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.593102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.593784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.594064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.594336 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.594972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.596893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.626446 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1446/2894] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:23.631064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.641867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3621a6-ad33-4061-bf10-2ef822ade0f2 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.708612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.709232 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.709509 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.709804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.710446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.712373 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.736067 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1449/2895] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:23.740880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.752009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43de9cf-8b4c-4499-a203-a2aaa441ead4 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.757704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.758320 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.758602 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.758872 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.759525 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.761148 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.776157 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1447/2896] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:23.780663 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.792618 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add89b8-c1be-4eae-9777-ebcd32087b7b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.798024 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.798636 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.798915 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.799238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.799839 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.801591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc99d6e-48a1-4a1c-a260-182647acda59 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.812701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1450/2897] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:23.817862 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:23.827935 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6183-5c5a-4e24-8989-488032c19920 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:23.833419 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:23.834095 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.834374 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.834646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] PATH_INFO: `/v3/projects/7f3475b7af524daa97f9301766fd108b/users/ce415829ba124778b54ce6f934cac24b/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.850707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3475b7af524daa97f9301766fd108b, user_id=ce415829ba124778b54ce6f934cac24b, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:23.853573 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:23.867494 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1448/2898] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:23 2022] PUT /identity/v3/projects/7f3475b7af524daa97f9301766fd108b/users/ce415829ba124778b54ce6f934cac24b/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:23.877912 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:23.878200 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:23.878471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:23.907201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] MFA Rules not processed for user `ce415829ba124778b54ce6f934cac24b`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:23.908420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:23.964362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:23.977089 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1451/2899] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:23.981634 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.008533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:24.053549 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332f47d-fb4d-45b2-89e5-8c7e32bed9aa admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.058246 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.058853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.059134 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.059438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] PATH_INFO: `/v3/projects` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.060069 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.062044 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.091466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1449/2900] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:23 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:24.095623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.106818 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a88b40-1d09-4ac9-8ba4-7893549ec6f1 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.166449 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.167064 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.167395 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.167668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] PATH_INFO: `/v3/users` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.168350 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.170360 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.193800 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1452/2901] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:24 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:24.197893 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.209193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c220-387d-4e0b-8a78-424300dadf5d admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.213966 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.214566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.215303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.215303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.215726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.217331 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.228213 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1450/2902] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:24.232480 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.243015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6baba5-21bd-4567-a07f-7baa197ddc51 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.247851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.248516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.248797 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.249072 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] PATH_INFO: `/v3/projects/58ce89357d3d4883a482600ce86dad39/users/ae620f6f4e304b2694bba98d7bf1750a/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.266755 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ce89357d3d4883a482600ce86dad39, user_id=ae620f6f4e304b2694bba98d7bf1750a, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.270265 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.284355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1453/2903] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:24 2022] PUT /identity/v3/projects/58ce89357d3d4883a482600ce86dad39/users/ae620f6f4e304b2694bba98d7bf1750a/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:24.288714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.299312 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8729-7d26-4515-883d-9156030f675f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.367005 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.367533 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.367811 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.368081 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.368728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.370402 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2c186-cf14-4be4-85ae-1d18ea1ba63d admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.385774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1451/2904] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:33:24.387009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.399363 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79668e-4040-487c-9ade-96574d54525c admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.404474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.405175 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.405454 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.405758 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] PATH_INFO: `/v3/projects/58ce89357d3d4883a482600ce86dad39/users/ae620f6f4e304b2694bba98d7bf1750a/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.407774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ce89357d3d4883a482600ce86dad39, user_id=ae620f6f4e304b2694bba98d7bf1750a, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.410850 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.422539 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1454/2905] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:24 2022] PUT /identity/v3/projects/58ce89357d3d4883a482600ce86dad39/users/ae620f6f4e304b2694bba98d7bf1750a/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:24.433637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.433953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.434231 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.462808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] MFA Rules not processed for user `ae620f6f4e304b2694bba98d7bf1750a`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:24.464036 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:24.542834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1452/2906] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:24.555793 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:24.557746 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:24.619219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81979b7-2da7-4559-989f-a2ea97e2f847 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:24.623045 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:24.623670 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:24.623950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:24.624215 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:24.635030 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:24.636485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47789da4-6ffc-438b-a325-395ae32f4752 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:24.647331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1455/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:26.167016 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:26.189937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:26.235707 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef6eb2-5268-4d36-a5ae-61dd7bd21983 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:26.241806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:26.242539 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:26.242790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:26.243066 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] PATH_INFO: `/v3/users/ce415829ba124778b54ce6f934cac24b` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:26.244232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce415829ba124778b54ce6f934cac24b)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:26.246224 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:26.299483 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce415829ba124778b54ce6f934cac24b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.300159 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce415829ba124778b54ce6f934cac24b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.313527 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce415829ba124778b54ce6f934cac24b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.323009 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce415829ba124778b54ce6f934cac24b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.323667 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce415829ba124778b54ce6f934cac24b'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.336768 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1453/2908] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:26 2022] DELETE /identity/v3/users/ce415829ba124778b54ce6f934cac24b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:26.341274 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:26.341828 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:26.341925 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:26.362318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:26.364467 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:26.410578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:26.429114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1456/2909] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:26.439181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:26.440671 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:26.506138 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03aa52-734c-4279-9c75-57abe28f7dc2 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:26.509738 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:26.510345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:26.510616 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:26.510880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:26.520735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:26.522221 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:26.532404 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1454/2910] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:26.808499 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:26.809972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:26.823952 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3c78f-ae6c-49b3-a663-c3da3bacde14 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:26.828969 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:26.829908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:26.830105 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:26.830356 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] PATH_INFO: `/v3/projects/7f3475b7af524daa97f9301766fd108b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:26.831526 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3475b7af524daa97f9301766fd108b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:26.834050 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:26.881012 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3475b7af524daa97f9301766fd108b', 'request_id': 'req-b4945331-c0f3-4479-b698-5228c31ae1e8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.887104 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3475b7af524daa97f9301766fd108b', 'request_id': 'req-b4945331-c0f3-4479-b698-5228c31ae1e8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.896891 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3475b7af524daa97f9301766fd108b', 'request_id': 'req-b4945331-c0f3-4479-b698-5228c31ae1e8'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.908308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] The token cache is being invalidate because project 7f3475b7af524daa97f9301766fd108b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:26.908591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.909220 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:26.910805 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1457/2911] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:26 2022] DELETE /identity/v3/projects/7f3475b7af524daa97f9301766fd108b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:26.914756 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:26.924294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0dd2f-9380-45d1-a76f-58546f45cff3 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:26.964807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:26.983126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:26.983744 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:26.984017 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:26.984281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] PATH_INFO: `/v3/users/ae620f6f4e304b2694bba98d7bf1750a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:26.985400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae620f6f4e304b2694bba98d7bf1750a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:26.987409 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.033728 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae620f6f4e304b2694bba98d7bf1750a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.034376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae620f6f4e304b2694bba98d7bf1750a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.048629 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae620f6f4e304b2694bba98d7bf1750a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.057060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae620f6f4e304b2694bba98d7bf1750a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.057799 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-3981284c-1e89-4dd4-bcd1-c51616bf0d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae620f6f4e304b2694bba98d7bf1750a'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.070972 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1455/2912] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:26 2022] DELETE /identity/v3/users/ae620f6f4e304b2694bba98d7bf1750a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:27.353392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.354901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:27.378569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945331-c0f3-4479-b698-5228c31ae1e8 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.436913 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.437461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.437767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.438040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] PATH_INFO: `/v3/projects/58ce89357d3d4883a482600ce86dad39` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.439172 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ce89357d3d4883a482600ce86dad39)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.441092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.488324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ce89357d3d4883a482600ce86dad39', 'request_id': 'req-622db200-95f2-4221-a3fd-8908075058b0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.493949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ce89357d3d4883a482600ce86dad39', 'request_id': 'req-622db200-95f2-4221-a3fd-8908075058b0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.501738 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ce89357d3d4883a482600ce86dad39', 'request_id': 'req-622db200-95f2-4221-a3fd-8908075058b0'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.512872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] The token cache is being invalidate because project 58ce89357d3d4883a482600ce86dad39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:27.513161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.513841 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-622db200-95f2-4221-a3fd-8908075058b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:27.515332 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1458/2913] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:27 2022] DELETE /identity/v3/projects/58ce89357d3d4883a482600ce86dad39 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:27.521344 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.521888 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.521989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.542574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:27.543779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:27.606143 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1456/2914] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:27.614516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.614804 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.615079 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.623046 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:27.637802 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:27.639033 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:27.647752 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1459/2915] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:27.652073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.663223 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a79a9-14ab-468f-8f84-1bc8e8c60b8f None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.667787 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.668376 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.668653 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.668926 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.669567 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:27.669898 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.671507 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.683576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1457/2916] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:33:27.688048 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.697908 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587468-4a49-4cf9-8161-5d040887d875 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.702550 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.703145 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.703422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.703691 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.704321 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.706293 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.736266 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1460/2917] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:27 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:27.741105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.751264 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58dc1-64e6-449e-8ce2-66af06b916df admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.797110 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:27.810721 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.811327 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.811605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.811880 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.812502 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.814467 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.836518 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1458/2918] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:27 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:27.840665 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.852092 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61112cff-72c9-4242-ae9b-b45bb5abecba admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.856757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.857353 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.857687 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.857960 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.858626 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.860227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.871702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1461/2919] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:27.875746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.886061 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad795-e3da-45f0-90ba-56c3bdc2b1ac admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.891209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.891814 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.892094 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.892417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.893064 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.894747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090d60f-3996-40d9-b97e-ef8aa96f8e03 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.906510 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1459/2920] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:27.911286 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:27.921414 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48ced-823b-45b7-9d09-d9136ce6fedf admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:27.926714 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:27.927427 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.927749 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.928031 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] PATH_INFO: `/v3/projects/69e73c02a3d54f55b6b17a54da275fba/users/485196e9a66f42d1bf1c646789d64147/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.942533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e73c02a3d54f55b6b17a54da275fba, user_id=485196e9a66f42d1bf1c646789d64147, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:27.945132 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:27.960471 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1462/2921] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:27 2022] PUT /identity/v3/projects/69e73c02a3d54f55b6b17a54da275fba/users/485196e9a66f42d1bf1c646789d64147/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:27.970909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:27.971202 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:27.971479 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:27.999793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] MFA Rules not processed for user `485196e9a66f42d1bf1c646789d64147`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:28.001015 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:28.071243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1460/2922] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:28.075905 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.145219 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bae87-df8c-4afb-900f-bbeedc4b8593 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.150201 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.150706 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.150965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.151234 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.151867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.154041 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.181250 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1463/2923] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:28 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:28.185313 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.196933 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e83e3e-c497-4e4e-8983-81dee24e18fe None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.253599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.255098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.255379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.255654 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.256300 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.258286 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.280780 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1461/2924] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:28 2022] POST /identity/v3/users => generated 310 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:28.284723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.295352 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0869-cf36-4cca-8852-1d0ad6e755f7 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.300040 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.301244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.301523 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.301827 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.302468 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.304080 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.314645 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1464/2925] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:28.319429 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.329356 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b3123-f154-4687-955f-7596ce075b98 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.334189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.334834 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.335105 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.335372 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] PATH_INFO: `/v3/projects/a711035df9684f269ef833aea6ac12ed/users/a3c1185ab17e45049d34f796a79c93c6/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.350437 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a711035df9684f269ef833aea6ac12ed, user_id=a3c1185ab17e45049d34f796a79c93c6, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.353097 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.366626 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1462/2926] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:28 2022] PUT /identity/v3/projects/a711035df9684f269ef833aea6ac12ed/users/a3c1185ab17e45049d34f796a79c93c6/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:28.370604 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.381441 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffcfc0-3691-4dca-ad0f-4a15ca817761 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.439594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.440204 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.440482 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.440751 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.442413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.444019 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02cb3-0b0e-44a0-b3df-11e79ab0e4cd admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.453813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1465/2927] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:28.458432 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.468679 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb06d84-e699-4cff-8932-847261466125 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.475029 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.475680 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.475955 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.476225 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] PATH_INFO: `/v3/projects/a711035df9684f269ef833aea6ac12ed/users/a3c1185ab17e45049d34f796a79c93c6/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.478166 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a711035df9684f269ef833aea6ac12ed, user_id=a3c1185ab17e45049d34f796a79c93c6, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.480807 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.492126 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1463/2928] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:28 2022] PUT /identity/v3/projects/a711035df9684f269ef833aea6ac12ed/users/a3c1185ab17e45049d34f796a79c93c6/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:28.503178 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.503464 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.504393 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.525260 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:28.535933 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] MFA Rules not processed for user `a3c1185ab17e45049d34f796a79c93c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:28.537158 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:28.607648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1466/2929] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:28.619209 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.621045 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:28.634238 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:28.684173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522075db-fd3b-41a5-b052-a77f801898a9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.687793 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.688416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.688689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.688953 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.698303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.699725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.709657 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1464/2930] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:28.865707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:28.875519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861441-6ef2-4350-b4e0-deedab8c2dd4 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:28.879901 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:28.880527 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:28.880798 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:28.881066 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:28.952778 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:28.954280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a801e9d-4e9b-4f2a-9b69-c4716b4349b3 service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:28.964494 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1467/2931] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:38.408719 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:38.471889 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bd28-b3bf-467d-b899-7bf14f858101 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:38.477436 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:38.478093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:38.478370 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:38.478644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] PATH_INFO: `/v3/users/485196e9a66f42d1bf1c646789d64147` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:38.479760 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485196e9a66f42d1bf1c646789d64147)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:38.481725 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:38.530441 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485196e9a66f42d1bf1c646789d64147'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:38.531466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485196e9a66f42d1bf1c646789d64147'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:38.546399 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485196e9a66f42d1bf1c646789d64147'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:38.554447 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485196e9a66f42d1bf1c646789d64147'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:38.555174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485196e9a66f42d1bf1c646789d64147'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:38.568450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1465/2932] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:38 2022] DELETE /identity/v3/users/485196e9a66f42d1bf1c646789d64147 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:38.573147 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:38.573616 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:38.573742 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:38.593603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:38.594919 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:38.658584 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1468/2933] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:38.669074 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:38.670536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:38.706732 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:38.732708 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db630-afc0-425f-a9b7-e49890526639 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:38.736373 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:38.736979 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:38.737259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:38.737530 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:38.747783 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:38.749215 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:38.759615 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1466/2934] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:39.038970 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.040387 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:39.055481 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db547198-ff0c-4213-9766-0110a1b1181b None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:39.060203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:39.060845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.061127 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.061400 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] PATH_INFO: `/v3/projects/69e73c02a3d54f55b6b17a54da275fba` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.062556 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e73c02a3d54f55b6b17a54da275fba)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:39.064485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:39.098702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:39.115086 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e73c02a3d54f55b6b17a54da275fba', 'request_id': 'req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.120555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e73c02a3d54f55b6b17a54da275fba', 'request_id': 'req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.128198 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e73c02a3d54f55b6b17a54da275fba', 'request_id': 'req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.140084 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] The token cache is being invalidate because project 69e73c02a3d54f55b6b17a54da275fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:39.140372 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.141073 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.142644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1469/2935] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:39 2022] DELETE /identity/v3/projects/69e73c02a3d54f55b6b17a54da275fba => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:39.147362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.162130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c140a-71e9-486b-adb5-e780658b5baa service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:39.238864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:39.239342 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.239624 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.239904 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] PATH_INFO: `/v3/users/a3c1185ab17e45049d34f796a79c93c6` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.241153 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c1185ab17e45049d34f796a79c93c6)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:39.243158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:39.292714 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c1185ab17e45049d34f796a79c93c6'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.293283 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c1185ab17e45049d34f796a79c93c6'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.305632 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c1185ab17e45049d34f796a79c93c6'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.312746 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c1185ab17e45049d34f796a79c93c6'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.313375 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-059ea641-1839-4afe-9554-cc68ccb5a9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c1185ab17e45049d34f796a79c93c6'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.326558 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1467/2936] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:39 2022] DELETE /identity/v3/users/a3c1185ab17e45049d34f796a79c93c6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:39.601083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.602713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:39.617566 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfee6ee-91b5-4a28-a7f4-f84c26e4e72e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:39.678538 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:39.679161 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.679439 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.679707 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] PATH_INFO: `/v3/projects/a711035df9684f269ef833aea6ac12ed` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.680822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a711035df9684f269ef833aea6ac12ed)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:39.682757 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:39.730130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a711035df9684f269ef833aea6ac12ed', 'request_id': 'req-e4f4b591-27f6-492e-ba09-60b8d78512f2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.735563 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a711035df9684f269ef833aea6ac12ed', 'request_id': 'req-e4f4b591-27f6-492e-ba09-60b8d78512f2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.743485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a711035df9684f269ef833aea6ac12ed', 'request_id': 'req-e4f4b591-27f6-492e-ba09-60b8d78512f2'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.755357 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] The token cache is being invalidate because project a711035df9684f269ef833aea6ac12ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:39.755644 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.756294 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-e4f4b591-27f6-492e-ba09-60b8d78512f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:39.757813 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1470/2937] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:39 2022] DELETE /identity/v3/projects/a711035df9684f269ef833aea6ac12ed => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:39.762806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.763262 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.763367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.783976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:39.785963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:39.834332 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:39.849577 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1468/2938] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:39.857419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.857736 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.858025 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.878355 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:39.879769 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:39.887991 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1471/2939] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 12:33:39.892827 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.903726 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115608d-6f36-4b29-9f0f-55e6bf34176d None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:39.909077 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:39.909854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.910704 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.910997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.911644 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:39.911937 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:39.913535 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:39.925367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1469/2940] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 12:33:39.930767 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.940108 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e48491-e762-414c-b594-c779fb4e0ff7 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:39.945083 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:39.945701 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:39.945982 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:39.946248 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:39.946917 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:39.948807 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:39.976431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1472/2941] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:39 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:39.980559 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:39.991305 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e244ecb-4b52-48c6-8c9d-3197dcca9c3f admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:40.047526 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:40.048137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.048414 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.048688 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.049329 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:40.051892 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:40.074252 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1470/2942] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:39 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:40.078510 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:40.088978 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70861c5-e9e9-41c4-846a-eb2fd4fc5e66 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:40.093712 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:40.094322 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.094594 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.094870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.095516 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:40.097116 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:40.107956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1473/2943] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:40.111929 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:40.122909 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a115286-8961-4166-b498-9ace1a936250 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:40.127613 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:40.128207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.128485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.128871 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.129569 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:40.131208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f5b42-074c-482f-9bad-2b9667a5124e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:40.142443 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1471/2944] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:40.146548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:40.157139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2126-23c1-4501-952c-cc871f9bdabc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:40.162609 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:40.163296 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.163574 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.163851 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] PATH_INFO: `/v3/projects/e71a5b55655c46b780a801e574f1801e/users/8623215f886346df967c4193bf582dbe/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.166287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:40.182710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71a5b55655c46b780a801e574f1801e, user_id=8623215f886346df967c4193bf582dbe, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:40.185595 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:40.200486 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1474/2945] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:40 2022] PUT /identity/v3/projects/e71a5b55655c46b780a801e574f1801e/users/8623215f886346df967c4193bf582dbe/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:40.205417 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71134-8f41-40be-9fb8-cedb74682e20 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.205729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71134-8f41-40be-9fb8-cedb74682e20 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.206006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71134-8f41-40be-9fb8-cedb74682e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.227915 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-afc71134-8f41-40be-9fb8-cedb74682e20 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:40.229158 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-afc71134-8f41-40be-9fb8-cedb74682e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:40.293367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1472/2946] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:40.303123 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:40.304823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:40.364070 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b0916-62cb-4ade-94b4-d2b6a25a3850 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:40.367724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:40.368324 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:40.368597 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:40.368862 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:40.378308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:40.380077 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:40.389555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1475/2947] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:41.035646 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.036324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.036324 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.064843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] MFA Rules not processed for user `8623215f886346df967c4193bf582dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:41.066127 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:41.119661 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:41.139272 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1473/2948] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:41.143652 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.154253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee783-282e-4072-a658-510618860b1e service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.158953 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.159548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.159838 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.160103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.160724 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.162898 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.196435 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1476/2949] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:41 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:41.198499 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.210928 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5fc9f-8119-4cde-b9c9-1c53d3073667 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.269118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.269792 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.270080 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.270364 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.270989 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.272908 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.294199 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1474/2950] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:41 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:41.298243 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.309392 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d0299-f067-493f-87cd-2e0fa6378f7e admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.314445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.315049 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.315331 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.315600 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.316869 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.318517 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.330090 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1477/2951] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:41.334788 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.344759 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af2d25-4be6-464f-98c4-b6e1e0a38bfb admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.350538 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.351189 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.351466 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.351737 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] PATH_INFO: `/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0/users/4b60e2a386ae481282d72ad1ecde900d/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.367599 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0dcf0b93ab444e88c63f236e5dc09a0, user_id=4b60e2a386ae481282d72ad1ecde900d, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.370289 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.383438 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1475/2952] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:41 2022] PUT /identity/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0/users/4b60e2a386ae481282d72ad1ecde900d/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:41.387591 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.398433 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e57a7c-4241-4b20-8749-0d14a42ef1dc admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.455493 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.456101 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.456380 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.456648 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.457316 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.458947 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d98f-3d2e-48dd-b7d1-2a8bd9829e39 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.468605 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1478/2953] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:33:41.473321 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:41.483497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4acfbd-64fc-4d93-8829-2f145b7ec37e admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:41.489156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:41.489857 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:41.490137 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:41.490411 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] PATH_INFO: `/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0/users/4b60e2a386ae481282d72ad1ecde900d/roles/edae49896b104fddabf7a02644f5d914` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:41.492361 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0dcf0b93ab444e88c63f236e5dc09a0, user_id=4b60e2a386ae481282d72ad1ecde900d, role_id=edae49896b104fddabf7a02644f5d914)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:41.496945 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:41.507454 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1476/2954] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:41 2022] PUT /identity/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0/users/4b60e2a386ae481282d72ad1ecde900d/roles/edae49896b104fddabf7a02644f5d914 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:33:42.165318 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:42.165852 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:42.165965 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:42.196022 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] MFA Rules not processed for user `4b60e2a386ae481282d72ad1ecde900d`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:42.197263 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:42.236699 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:42.278612 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1479/2955] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:42.290481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:42.292243 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:42.353481 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228611c0-ac16-4eea-ac72-7abeb55867f5 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:42.357054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:42.357683 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:42.357964 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:42.358232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:42.413532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:42.416130 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:42.418480 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:42.428839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1477/2956] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:43.798240 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:43.799666 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:43.812531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe58c0-6b9a-44c1-aa47-9aac17e4fe24 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:43.816071 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:43.817723 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:43.817992 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:43.818259 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:43.827304 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:43.828725 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:43.839343 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1480/2957] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:33:52.423963 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=2 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:53.835095 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=18 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:56.984534 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:57.050452 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6748081-08cb-435f-9cf5-fe9e622886c4 service cinder] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:57.056278 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:57.056922 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:57.057193 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:57.057463 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] PATH_INFO: `/v3/users/8623215f886346df967c4193bf582dbe` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:57.058895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8623215f886346df967c4193bf582dbe)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:57.060813 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:57.108729 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8623215f886346df967c4193bf582dbe'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.109294 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8623215f886346df967c4193bf582dbe'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.121179 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8623215f886346df967c4193bf582dbe'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.128605 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8623215f886346df967c4193bf582dbe'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.129180 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8623215f886346df967c4193bf582dbe'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.142764 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1478/2958] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:56 2022] DELETE /identity/v3/users/8623215f886346df967c4193bf582dbe => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:57.327016 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:57.339139 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa6958-314a-48d1-bfc6-462c5d4d6d1f service cinder] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:57.407119 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:57.407681 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:57.407955 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:57.408227 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] PATH_INFO: `/v3/projects/e71a5b55655c46b780a801e574f1801e` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:57.409376 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71a5b55655c46b780a801e574f1801e)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:57.411334 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:57.459299 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71a5b55655c46b780a801e574f1801e', 'request_id': 'req-aaea533f-dde1-49e1-9960-7fc7181ed137'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.465121 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71a5b55655c46b780a801e574f1801e', 'request_id': 'req-aaea533f-dde1-49e1-9960-7fc7181ed137'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.473578 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71a5b55655c46b780a801e574f1801e', 'request_id': 'req-aaea533f-dde1-49e1-9960-7fc7181ed137'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.485445 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] The token cache is being invalidate because project e71a5b55655c46b780a801e574f1801e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:57.485773 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.486446 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.487936 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1481/2959] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:57 2022] DELETE /identity/v3/projects/e71a5b55655c46b780a801e574f1801e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:57.492806 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:57.494762 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:57.508997 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8757b0-cb49-4760-ab83-caf1d2467918 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:57.564767 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:57.565387 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:57.565689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:57.565968 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] PATH_INFO: `/v3/users/4b60e2a386ae481282d72ad1ecde900d` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:57.567108 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b60e2a386ae481282d72ad1ecde900d)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:57.569853 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:57.592131 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:57.618840 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b60e2a386ae481282d72ad1ecde900d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.619497 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b60e2a386ae481282d72ad1ecde900d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.633197 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b60e2a386ae481282d72ad1ecde900d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.640844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b60e2a386ae481282d72ad1ecde900d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.641576 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-e8f14686-2d48-4c6a-b03f-da7705c80c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b60e2a386ae481282d72ad1ecde900d'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.654362 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1479/2960] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:33:57 2022] DELETE /identity/v3/users/4b60e2a386ae481282d72ad1ecde900d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:57.836251 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:57.837741 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:57.851585 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea533f-dde1-49e1-9960-7fc7181ed137 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:57.911317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:57.912698 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:57.913151 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:57.913425 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] PATH_INFO: `/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:57.914603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0dcf0b93ab444e88c63f236e5dc09a0)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:57.916531 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:57.951840 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:57.966009 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0dcf0b93ab444e88c63f236e5dc09a0', 'request_id': 'req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.971569 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0dcf0b93ab444e88c63f236e5dc09a0', 'request_id': 'req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.979622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0dcf0b93ab444e88c63f236e5dc09a0', 'request_id': 'req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.991765 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] The token cache is being invalidate because project f0dcf0b93ab444e88c63f236e5dc09a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:33:57.992052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.992690 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-b58d0421-ab36-46a9-be16-01a2c0ec9aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:33:57.994244 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1482/2961] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:33:57 2022] DELETE /identity/v3/projects/f0dcf0b93ab444e88c63f236e5dc09a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:58.000297 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.000986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.000986 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.021548 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:58.022790 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:58.087665 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1480/2962] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:58.095655 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.095942 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.096212 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.117078 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:58.118317 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:58.126231 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1483/2963] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:58.131014 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.140843 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3de21-a268-41ea-a45a-0f96aa1d415a None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.145505 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.146781 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.147059 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.147341 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.147984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87996) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 12:33:58.148282 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.149924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.162006 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1481/2964] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 6 12:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 12:33:58.166599 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.175777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f764762-2f1c-4caf-9eb5-9a99c9cbd066 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.180882 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.181474 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.181774 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.182042 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] PATH_INFO: `/v3/projects` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.182685 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.184680 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.213614 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1484/2965] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 6 12:33:58 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:58.218060 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.228498 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae244f6-ae24-4964-8690-58dcc9137f64 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.286675 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.287932 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.288208 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.288476 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.289102 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.291055 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.312118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1482/2966] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 6 12:33:58 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 12:33:58.316603 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.327235 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167040c0-924e-4871-8918-8fe8a4071228 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.332052 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.332656 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.332934 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.333205 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] PATH_INFO: `/v3/roles` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.334630 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.336238 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.346268 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1485/2967] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 12:33:58.351054 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.361173 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4202de3-d91c-4e35-a238-d5104e1b96c9 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.366245 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.366974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.367257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.367531 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.368174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.369812 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79587c6-9b00-46a0-a2aa-efbda3395c3c admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.382107 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1483/2968] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 6 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 12:33:58.386422 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.397845 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1668a-d215-4396-8c29-bcb2129ef696 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.402973 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.403627 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] REQUEST_METHOD: `PUT` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.404160 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.404431 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] PATH_INFO: `/v3/projects/f66b2abf1d934c5c97ab2aedd9932baa/users/7c53f8ae37b144da81631312a4a43274/roles/0da2f51a570245b1acf832a45de984cc` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.420622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66b2abf1d934c5c97ab2aedd9932baa, user_id=7c53f8ae37b144da81631312a4a43274, role_id=0da2f51a570245b1acf832a45de984cc)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.423452 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.436932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1486/2969] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 6 12:33:58 2022] PUT /identity/v3/projects/f66b2abf1d934c5c97ab2aedd9932baa/users/7c53f8ae37b144da81631312a4a43274/roles/0da2f51a570245b1acf832a45de984cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:33:58.448181 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] REQUEST_METHOD: `POST` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.448543 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.448885 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.459554 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:33:58.482133 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.auth.core [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] MFA Rules not processed for user `7c53f8ae37b144da81631312a4a43274`. Rule list: `[]` (Enabled: `True`). {{(pid=87996) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:33:58.483379 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:58.551471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1484/2970] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:33:58.562087 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:33:58.563822 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:33:58.625289 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e64ec-ccd3-4ec0-90bb-411bf6bf042b admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:33:58.629184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccafefb49d0c4727bad188927ae5bf7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:33:58.629829 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:33:58.630118 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:33:58.630923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:33:58.640068 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:33:58.641520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3708-acfc-42fa-8274-7c30b4cc4d0c service cinder] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:33:58.651863 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1487/2971] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:34:08.548363 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=30 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:34:08.646611 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:34:08.765371 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:34:08.827930 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2b514-6d6e-44c7-8335-a6a36f4c6de4 None None] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:34:08.832562 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:34:08.833174 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:34:08.833450 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:34:08.833747 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] PATH_INFO: `/v3/users/7c53f8ae37b144da81631312a4a43274` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:34:08.834864 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c53f8ae37b144da81631312a4a43274)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:34:08.837334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:34:08.884808 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c53f8ae37b144da81631312a4a43274'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:08.885471 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c53f8ae37b144da81631312a4a43274'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:08.899170 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c53f8ae37b144da81631312a4a43274'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:08.907335 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c53f8ae37b144da81631312a4a43274'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:08.908119 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c53f8ae37b144da81631312a4a43274'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:08.921167 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1485/2972] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:34:08 2022] DELETE /identity/v3/users/7c53f8ae37b144da81631312a4a43274 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:34:08.925864 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] REQUEST_METHOD: `POST` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:34:08.926156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:34:08.926436 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:34:08.947520 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.auth.core [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] MFA Rules not processed for user `f2212262a5dc4cac8d774b88f87a9ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=87997) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 12:34:08.948743 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:34:09.007667 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1488/2973] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 6 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 12:34:09.018352 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:34:09.019784 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:34:09.078156 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaeff1-0101-45df-816d-ac9420f01161 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:34:09.081727 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:34:09.082333 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:34:09.082604 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:34:09.082869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:34:09.092903 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:34:09.094367 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:34:09.103876 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1486/2974] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:34:09.375419 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:34:09.376946 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:34:09.389831 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb903f-bf84-4db9-88a4-43dd86f95a65 None None] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:34:09.394458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:34:09.395088 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:34:09.395362 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:34:09.395623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] PATH_INFO: `/v3/projects/f66b2abf1d934c5c97ab2aedd9932baa` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:34:09.396782 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66b2abf1d934c5c97ab2aedd9932baa)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:34:09.398713 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:34:09.446710 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66b2abf1d934c5c97ab2aedd9932baa', 'request_id': 'req-48a4d568-97e0-4ed2-bc40-f5300663bf30'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:09.452107 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66b2abf1d934c5c97ab2aedd9932baa', 'request_id': 'req-48a4d568-97e0-4ed2-bc40-f5300663bf30'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:09.459408 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66b2abf1d934c5c97ab2aedd9932baa', 'request_id': 'req-48a4d568-97e0-4ed2-bc40-f5300663bf30'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:09.471187 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] The token cache is being invalidate because project f66b2abf1d934c5c97ab2aedd9932baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87997) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:34:09.471475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:09.472103 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:34:09.473972 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1489/2975] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:34:09 2022] DELETE /identity/v3/projects/f66b2abf1d934c5c97ab2aedd9932baa => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:34:19.099537 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:34:19.469214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:34:47.192260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:34:47.193763 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:34:47.270303 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb447b69-a62a-47b4-b8ce-653dede91eec service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:34:47.329858 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:34:47.330461 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:34:47.330735 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:34:47.331003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:34:47.332730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:34:47.344532 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:34:47.346198 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:34:47.356601 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1487/2976] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:34:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:34:57.351741 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=58 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:35:29.373461 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:29.374900 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:29.435968 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d568-97e0-4ed2-bc40-f5300663bf30 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:29.439534 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:29.440420 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:29.440694 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:29.440956 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:29.442932 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:29.514020 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:29.515358 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:29.526513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1490/2977] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 12:35:30.717796 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:30.719235 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:30.730582 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322e74-5ee7-4e47-91f1-3cd5b2065495 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:30.734416 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:30.735003 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:30.735281 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:30.735553 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:30.737232 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:30.751345 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:30.752860 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:30.764770 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1488/2978] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:39.521628 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=54 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:35:40.759257 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=6 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:35:42.738983 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:42.747817 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d994-ce78-464d-90dd-5c8c53778fda service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:42.752264 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:42.752861 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:42.753140 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:42.753413 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:42.755239 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:42.764921 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:42.767312 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:42.775582 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1491/2979] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:42.905195 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:42.915691 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376b235-8c1d-4956-a1c9-f9cc9d55d6c4 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:42.919234 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:42.920115 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:42.920407 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:42.920730 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:42.923912 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:42.937673 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:42.939334 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:42.948574 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1489/2980] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:43.036489 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:43.051522 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18033a0-bffb-44ff-848b-d36c91ff9384 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:43.055870 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:43.056548 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:43.056923 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:43.057308 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:43.059442 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:43.075156 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:43.076615 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:43.088485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1492/2981] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:43.153004 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:43.154514 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:43.169398 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb65b-c2ff-4e3a-af3f-4c524257f4d1 service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:43.174984 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:43.175566 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:43.175839 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:43.176103 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:43.177774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:43.190154 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:43.191779 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:43.202073 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1490/2982] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:46.126443 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:35:46.127877 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:46.143664 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229f0f9-7b2a-493e-a80e-05f172abb783 service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:35:46.149536 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:35:46.150199 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:35:46.150475 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:35:46.150745 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:35:46.152472 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:35:46.166130 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:35:46.167722 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:35:46.180463 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1493/2983] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:35:53.197686 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=12 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:35:56.175448 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=18 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:37:30.631795 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:30.641536 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd4046-8b9b-490b-a722-cd924e0c86ca service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:30.645504 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:30.646136 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:30.646420 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:30.646692 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:30.648435 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:30.707400 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:30.708838 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:30.717623 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1491/2984] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:37:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:37:31.874990 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:31.876731 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:31.892001 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266af00-cfb5-47b3-b442-80c72be982a6 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:31.899998 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:31.900623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:31.900907 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:31.901180 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:31.903189 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:31.917623 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:31.920233 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:31.931881 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1494/2985] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:37:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:37:34.524570 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:34.540555 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcb07b-06c9-494a-9eb9-07e434cddc0f service neutron] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:34.546469 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9260fd9f9c524d20a9025595a53ade4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:34.547337 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:34.547621 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:34.548118 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:34.550024 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:34.570689 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:34.572316 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:34.587098 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1492/2986] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:37:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:37:35.846253 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:35.855962 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aae7d-10e5-4555-b763-409e7056aae3 service nova] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:35.859650 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52624a6d02e43aeb9bb33e61f1bfac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:35.860247 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:35.860525 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:35.860794 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:35.862533 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:35.928203 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:35.929668 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:35.940246 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1495/2987] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:37:42.096774 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:42.099884 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:42.120415 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132abb11-25e0-4918-917e-2d6f33c5babd service nova] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:42.125583 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:42.126302 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] REQUEST_METHOD: `GET` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:42.126578 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:42.126854 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:42.128913 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:42.140895 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:42.142520 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:42.156226 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1493/2988] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:37:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:37:42.188184 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:42.189747 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:42.203458 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b771-98d2-40e7-ade9-037453bea23d service neutron] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:42.208455 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:42.209114 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:42.209398 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:42.209759 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] PATH_INFO: `/v3/users/1102506cce6d4bd68d1af92fa8811fe3` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:42.210894 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1102506cce6d4bd68d1af92fa8811fe3)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:42.212823 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:42.266977 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1102506cce6d4bd68d1af92fa8811fe3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.276844 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1102506cce6d4bd68d1af92fa8811fe3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.294096 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1102506cce6d4bd68d1af92fa8811fe3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.294777 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1102506cce6d4bd68d1af92fa8811fe3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.302519 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1102506cce6d4bd68d1af92fa8811fe3'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.308764 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1496/2989] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:37:42 2022] DELETE /identity/v3/users/1102506cce6d4bd68d1af92fa8811fe3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:37:42.495078 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:42.508649 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198b4d3-eb79-46b0-b6c9-cb1ce2529822 service placement] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:42.572637 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:42.573259 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:42.573540 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:42.573844 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] PATH_INFO: `/v3/projects/4e13494666564124bdf4007277a515d6` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:42.574976 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e13494666564124bdf4007277a515d6)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:42.576899 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:42.625869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e13494666564124bdf4007277a515d6', 'request_id': 'req-d1ad43d3-5878-443b-bc94-a782bc0d9c08'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.634974 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e13494666564124bdf4007277a515d6', 'request_id': 'req-d1ad43d3-5878-443b-bc94-a782bc0d9c08'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.640995 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e13494666564124bdf4007277a515d6', 'request_id': 'req-d1ad43d3-5878-443b-bc94-a782bc0d9c08'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.652958 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] The token cache is being invalidate because project 4e13494666564124bdf4007277a515d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:37:42.653244 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.653924 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.655403 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1494/2990] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:37:42 2022] DELETE /identity/v3/projects/4e13494666564124bdf4007277a515d6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:37:42.659872 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:42.661320 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:42.674622 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cabb6b-76a1-4709-bd81-41c3e7df0a01 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:42.723242 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:37:42.735005 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:42.735735 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:42.736015 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:42.736287 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] PATH_INFO: `/v3/users/ee4674549863473c9edabc6bf941982b` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:42.737513 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4674549863473c9edabc6bf941982b)` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:42.740000 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:42.787485 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4674549863473c9edabc6bf941982b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.794867 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4674549863473c9edabc6bf941982b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.808280 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4674549863473c9edabc6bf941982b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.808949 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4674549863473c9edabc6bf941982b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.817075 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.notifications [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4674549863473c9edabc6bf941982b'} {{(pid=87997) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:42.823177 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1497/2991] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 6 12:37:42 2022] DELETE /identity/v3/users/ee4674549863473c9edabc6bf941982b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 12:37:43.009169 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Authenticating user token {{(pid=87996) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:37:43.010900 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.fernet_utils [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87996) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:37:43.018877 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:37:43.025491 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad43d3-5878-443b-bc94-a782bc0d9c08 admin admin] Validating token access rules against request {{(pid=87996) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:37:43.086971 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2212262a5dc4cac8d774b88f87a9ec9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff9660c7d5b94c33b26aef6fd109810a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87996) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:37:43.087592 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:37:43.087869 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:37:43.088142 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] PATH_INFO: `/v3/projects/69d07392210b4dca8703e02c4e09153a` {{(pid=87996) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:37:43.089260 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d07392210b4dca8703e02c4e09153a)` {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:37:43.091214 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] RBAC: Authorization granted {{(pid=87996) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:37:43.139485 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d07392210b4dca8703e02c4e09153a', 'request_id': 'req-1448d037-e9b7-453a-bc6a-0daa9b741c62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:43.147117 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d07392210b4dca8703e02c4e09153a', 'request_id': 'req-1448d037-e9b7-453a-bc6a-0daa9b741c62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:43.152883 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d07392210b4dca8703e02c4e09153a', 'request_id': 'req-1448d037-e9b7-453a-bc6a-0daa9b741c62'} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:43.165207 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] The token cache is being invalidate because project 69d07392210b4dca8703e02c4e09153a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87996) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 12:37:43.165493 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:43.166168 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG keystone.notifications [None req-1448d037-e9b7-453a-bc6a-0daa9b741c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87996) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 12:37:43.167652 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: [pid: 87996|app: 0|req: 1495/2992] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 6 12:37:43 2022] DELETE /identity/v3/projects/69d07392210b4dca8703e02c4e09153a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 12:37:52.819634 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:37:53.162093 dv-cinder-856041-1-iscsi devstack@keystone.service[87996]: DEBUG dbcounter [-] [87996] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87996) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 12:39:56.202214 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Authenticating user token {{(pid=87997) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 12:39:56.203702 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:39:56.347880 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8ed51-696b-48ed-8eda-772bdfbf4c88 admin admin] Validating token access rules against request {{(pid=87997) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 12:39:56.409950 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fd64b822c934dada9d93d8179e374ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5695f5deeba84fb2b220872b905c6201', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87997) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 12:39:56.410834 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] REQUEST_METHOD: `GET` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 12:39:56.411613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] SCRIPT_NAME: `/identity` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 12:39:56.411613 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87997) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 12:39:56.413109 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.fernet_utils [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87997) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 12:39:56.425447 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 12:39:56.427098 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993373ed-44bb-48ff-ae03-e59fc1014503 service placement] RBAC: Authorization granted {{(pid=87997) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 12:39:56.437555 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: [pid: 87997|app: 0|req: 1498/2993] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 6 12:39:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 12:40:06.433047 dv-cinder-856041-1-iscsi devstack@keystone.service[87997]: DEBUG dbcounter [-] [87997] Writing DB stats keystone:SELECT=87 {{(pid=87997) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}